Automatic visual sensemaking approach for the behavioral understanding of beef cattle
Date
2022-12
Authors
Sourav, Md Abdullah All
Major Professor
Advisor
Peschel, Joshua
Ramirez, Brett
Hansen, Stephanie
Koziel, Jacek
Johnson, Anna
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Computer vision has been extensively used for livestock monitoring in recent years. Most research in this domain focuses on the performance evaluation of algorithms used to analyze visual sensing data collected from specially designed environments. There is a scope for finding the best combination of hardware placement to collect data, evaluate algorithm performance, and develop a set of recommendations for livestock behavioral data analysis. Thus, this dissertation seeks to find the appropriate automatic visual sensemaking approach for achieving a behavioral understanding of steer in a confined feeding operation. The camera configuration needed for observing steer, the algorithm required for behavioral data analysis, and the impact of the pen environment on behavioral analysis were investigated to answer the question.
Data collection with a sensor or camera is the first part of the complete workflow. This study sought to determine the optimal camera placement locations in a confined steer feeding operation. Measurements of cattle pens were used to create a 3D farm model using Blender 3D computer graphic software. A method was developed to calculate the camera coverage in a 3D farm environment, and a genetic algorithm-based model was designed to find optimal placements of a multi-camera and multi-pen setup. The algorithm's objective was to maximize the multi-camera coverage while minimizing the budget. Two different optimization methods involving multiple cameras and pen combinations were used. The results demonstrated the applicability of the genetic algorithm in achieving maximum coverage and thereby enhancing the quality of the livestock visual-sensing data. The algorithm also provided the top 25 solutions for each camera and pen combination with a maximum coverage difference of less than 3.5% between them. It offers other alternative options when one solution is not practical for any reason, for example, the unavailability of a power outlet.
The detection and tracking of individual steers are essential steps for a computer vision-based steer monitoring system. An attempt was made to detect, track, and classify steer’s behaviors using state-of-the-art deep learning object detection methods. We trained Faster RCNN ResNet-50, Faster RCNN ResNet-101, Faster RCNN Inception V2, SSD MobileNet V1, YOLO V3, and YOLO V5 with our custom dataset containing 3305 images of the pen to identify and localize steers. A simple online tracker was used to track the steer and a location-based function was used to initiate re-track when tracking is lost. The current behaviors of the cattle were classified based on spatial location, travel path, and body shape detected by the object detection algorithm. All Faster RCNN variants, YOLO V3, and YOLO V5 models achieved more than 99% F1 score in detection. The result also showed that eating, standing, and laying behaviors are correctly classified in all models with around 90% accuracy. However, misclassification between standing and laying behaviors was observed in several cases due to similarities in steer body image between standing and laying and lack of visual details. The promising performance of combining the Faster RCNN and YOLO-based model with the tracking algorithm demonstrates the feasibility of deep learning-based object detection and tracking. The behavior detection model also provided time spent on each behavior by each steer in given pen videos. In addition, the time-series behavioral data analysis method to understand the current behaviors were also summarized.
Contributions of this dissertation include (a) it is the first focused study on the role of hardware placement and behavior detection algorithm integration in livestock monitoring, (b) a novel automated visual system for behavior monitoring in a confined beef cattle operation, and (c) a set of recommendations provided for livestock behavior monitoring with visual sensing and sensemaking.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
dissertation