Panoptic Segmentation of Individual Pigs for Posture Recognition

Abstract

<jats:p>Behavioural research of pigs can be greatly simplified if automatic recognition systems are used. Systems based on computer vision in particular have the advantage that they allow an evaluation without affecting the normal behaviour of the animals. In recent years, methods based on deep learning have been introduced and have shown excellent results. Object and keypoint detector have frequently been used to detect individual animals. Despite promising results, bounding boxes and sparse keypoints do not trace the contours of the animals, resulting in a lot of information being lost. Therefore, this paper follows the relatively new approach of panoptic segmentation and aims at the pixel accurate segmentation of individual pigs. A framework consisting of a neural network for semantic segmentation as well as different network heads and postprocessing methods will be discussed. The method was tested on a data set of 1000 hand-labeled images created specifically for this experiment and achieves detection rates of around 95% (F1 score) despite disturbances such as occlusions and dirty lenses.</jats:p>

Practices and Applications of Convolutional Neural Network-Based Computer Vision Systems in Animal Farming: A Review

“…Additionally, temporal acquisition of outdoor images also introduced variations of background [ 54 ]. Animal confinement facilities and facility arrangements vary across farms and production systems [ 59 , 60 ], therefore, moving cameras or placing multiple cameras in different locations within the same farm/facility can also capture various backgrounds contributing to data diversity [ 61 , 62 ]. Recording images under different lighting conditions (e.g., shadow, sunlight, low/medium/high light intensities, etc.)…”

“…Labels that are input into model training influence what patterns models learn from an image. For some applications of animal species recognition, professional labeling knowledge in animals may not be required since labelers only needed to distinguish animals from their respective backgrounds [ 59 ]. However, as for behavior recognition, animal scientists were generally required to assist in behavior labeling, because labels of animal behaviors should be judged accurately by professional knowledge [ 102 ].…”

“…Adding noises includes blurring images with filters (e.g., median filters [ 100 ]), introducing additional signals (e.g., Gaussian noises [ 100 ]) onto images, adding random shapes (e.g., dark polygons [ 70 ]) onto images, and padding pixels around boundaries of images [ 100 ]. Changing color space contains altering components (e.g., hues, saturation, brightness, and contrast) in HSV space or in RGB space [ 88 , 216 ], switching illumination styles (e.g., simulate active infrared illumination [ 59 ]), and manipulating pixel intensities by multiplying them with various factors [ 121 ]. All these may create variations during training and make models avoid repeatedly learning homogenous patterns and thus reduced overfitting.…”

“…s is segment. [59] Specific metrics for pose estimation PCKh ∑ , − − ℎ ℎ × ℎ _ Evaluation of correctly detected key points based on sizes of object heads. n is number of images; , is the j th predicted key points in the i th image; is the j th key points of ground truth in the i th image; and ℎ _ is the length of heads in the i th image [216]…”

Learn to Train: Improving Training Data for a Neural Network to Detect Pecking Injuries in Turkeys

“…Xu et al described a machine-learning-based quadcopter vision system for the classification and counting of sheep and cattle, where each animal on the processed images was labeled with semantic segmentation [ 25 ]. A combined method of instance and semantic segmentation was investigated to trace the contours of pigs by Brünger et al [ 26 ]. Their method provided a pixel-accurate segmentation of the pig and achieved very good detection rates (F1-score: 95%) when tested on the used data set.…”