TSML: A New Pig Behavior Recognition Method Based on Two-Stream Mutual Learning Network

Abstract

Changes in pig behavior are crucial information in the livestock breeding process, and automatic pig behavior recognition is a vital method for improving pig welfare. However, most methods for pig behavior recognition rely on human observation and deep learning. Human observation is often time-consuming and labor-intensive, while deep learning models with a large number of parameters can result in slow training times and low efficiency. To address these issues, this paper proposes a novel deep mutual learning enhanced two-stream pig behavior recognition approach. The proposed model consists of two mutual learning networks, which include the red–green–blue color model (RGB) and flow streams. Additionally, each branch contains two student networks that learn collaboratively to effectively achieve robust and rich appearance or motion features, ultimately leading to improved recognition performance of pig behaviors. Finally, the results of RGB and flow branches are weighted and fused to further improve the performance of pig behavior recognition. Experimental results demonstrate the effectiveness of the proposed model, which achieves state-of-the-art recognition performance with an accuracy of 96.52%, surpassing other models by 2.71%.

A Short Video Classification Framework Based on Cross-Modal Fusion

“…We stem these textual features and remove words using the standard contextual attention mechanism. We use the filtering mechanism to perform a zero-fill operation for those that are not long enough, and directly truncate those that are too long to remove interference [37,38]. For the visual features of the video, we use the Timesformer method to extract video features using the spatio-temporal self-attention mechanism.…”

Pig Movement Estimation by Integrating Optical Flow with a Multi-Object Tracking Model

“…Tu et al [27] achieved pig behavior recognition based on tracking algorithms where the behavior identification of each pig was based on tracking results. Hao et al [11] proposed a deep mutual learning enhanced two-stream method consisting of two mutual learning networks for identifying pig behaviors. In their approach, two mutual learning networks were able to extract rich appearance and motion features, improving performance.…”

“…Broadly, two predominant methodologies for estimating pig movement have emerged. The first approach leverages behavior recognition algorithms [9][10][11][12] to classify various pig activities, encompassing lying, walking, sitting, standing, drinking, etc. The second approach harnesses tracking algorithms [8,13,14] within the domain of computer vision to monitor the positions of individual pigs and subsequently assess pig movement using the center point of bounding boxes.…”