Instance Segmentation with Mask R-CNN Applied to Loose-Housed Dairy Cows in a Multi-Camera Setting

Abstract

With increasing herd sizes came an enhanced requirement for automated systems to support the farmers in the monitoring of the health and welfare status of their livestock. Cattle are a highly sociable species, and the herd structure has important impact on the animal welfare. As the behaviour of the animals and their social interactions can be influenced by the presence of a human observer, a camera based system that automatically detects the animals would be beneficial to analyse dairy cattle herd activity. In the present study, eight surveillance cameras were mounted above the barn area of a group of thirty-six lactating Holstein Friesian dairy cows at the Chamber of Agriculture in Futterkamp in Northern Germany. With Mask R-CNN, a state-of-the-art model of convolutional neural networks was trained to determine pixel level segmentation masks for the cows in the video material. The model was pre-trained on the Microsoft common objects in the context data set, and transfer learning was carried out on annotated image material from the recordings as training data set. In addition, the relationship between the size of the used training data set and the performance on the model after transfer learning was analysed. The trained model achieved averaged precision (Intersection over union, IOU = 0.5) 91% and 85% for the detection of bounding boxes and segmentation masks of the cows, respectively, thereby laying a solid technical basis for an automated analysis of herd activity and the use of resources in loose-housing.

SheepInst: A High-Performance Instance Segmentation of Sheep Images Based on Deep Learning

“…Qiao et al [ 19 ] proposed an instance segmentation method based on Mask R-CNN deep learning framework for solving the problem of cattle segmentation and contour extraction in the real environment. The authors [ 20 ] proposed the instance segmentation with Mask R-CNN of dairy cows to analyze dairy cattle herd activity in a multi-camera setting. Dohmen et al [ 21 ] applied Mask R-CNN algorithm to segment the regions of heifers in the images to support body mass prediction.…”

Study of a QueryPNet Model for Accurate Detection and Segmentation of Goose Body Edge Contours

“…Based on this, it is gradually being applied in animal husbandry farming. Non-invasive monitoring methods using sensors and cameras to acquire data and then processing the data through computer vision is a research hotspot in precision animal husbandry today [ 9 ]. The acquisition of livestock images using cameras and other means, followed by automated monitoring with the aid of computer vision, can result in substantial labor and equipment cost savings.…”

Enhanced Mask R-CNN for herd segmentation

“…But, the quality of the image segmentation can become a challenge due to both internal and external factors such as poor illumination and heterogeneous background [17] . To address the segmentation challenges iterated in this study, different contributions have been made in different studies using convolutional neural networks (CNN) based approaches with powerful abilities to learn the spatial-rich and semantic-information features [18][19][20][21] . In this study, an enhanced Mask Region-based-Convolutional Neural Network (Mask R-CNN) is proposed for herd segmentation so that an accurate segmentation can be accomplished in an environment full of complex backgrounds.…”