Video analytics for lameness detection in dairy cattle: Effects of background removal and deep image matting on farm videos
Date
2023-05
Authors
Ankita, Ankita
Major Professor
Advisor
Pandey, Santosh
Shearer, Jan
Kim, Jaeyoun
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In computer vision, it is often challenging to identify moving objects where there can be ambiguity between the foreground and background. This issue becomes prominent in videos recorded in real-world environments, such as those related to the pose estimation of unconstrained cows in the farm premises. For example, cow videos are often recorded of cows in different farm settings where the background has various types of external objects, equipment, boundary walls, fences, other animals, and humans. To address the variability in background, a well-studied area of research in computer vision is called image matting which involves the image processing tasks of removing the background and accurately estimating the foreground. Previous image matting algorithms were computationally simple because they used low level features and did not incorporate high-level contextual information. In recent years, image matting has evolved to incorporate deep learning techniques with a goal to address the ongoing limitations of conventional image matting. This method of background removal is called deep image matting which has shown promising results for human applications, such as to identify the facial features and body signatures of individuals in a crowd setting. Deep image matting is a recent development in computer vision that employs a more practical and efficient implementation and requires smaller datasets having complex background colors and textures with similar foreground and background colors.
In this thesis, I explored the feasibility of deep image matting for the dairy industry specifically, to explore the method of deep image matting to remove background in videos of cows while walking for accurate pose estimation and the eventual prediction of abnormal posture of walking cows which is indicative of lameness. Pose estimation has been widely studied on human images and videos to extract useful information and signatures from facial and body landmarks but has been largely unexplored in the dairy industry. Our team collected side-view videos of cows while walking and trained the deep neural network model with images with and without background. Our research lab has access to the Iowa State University Dairy Teaching Farm and our collaborator, Brian Pingsterhaus who is a professional hoof trimmer. Around 20 videos have been used for training my deep learning model. Each video was split into 20 frames. The labeled frames from the videos were further split into train and test datasets. In this case, 95% of the labeled frames were used as the train dataset and 5% were used as the test dataset to evaluate the model. Training the labeled dataset consisted of a relatively small number of labeled frames using ImageNet. The feature detector architecture has been derived from DeeperCut a human pose estimation algorithm.
The trained deep learning model was evaluated by measuring its performance in terms of mean Euclidean error (MAE), which is proportional to the average root mean square error. The MAE was calculated by measuring the difference between the labels (or key points) marked manually on frames and the labels applied by the model. New videos were then analyzed from the trained model after reviewing the output CSV file containing information about the key points predicted by the model. My results are intended to show whether the combined approach of DeepLabCut and deep image matting leads to enhanced model performance which is evaluated through pixel errors and the locations of key points on the cows’ body. My results demonstrate that the background removal step may be beneficial in some situations to distinguish the foreground from the background, however this additional step requires extra time and effort.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
thesis