Structured Context Enhancement Network for Mouse Pose Estimation

Abstract

Automated analysis of mouse behaviours is crucial for many applications in neuroscience. However, quantifying mouse behaviours from videos or images remains a challenging problem, where pose estimation plays an important role in describing mouse behaviours. Although deep learning based methods have made promising advances in mouse or other animal pose estimation, they cannot properly handle complicated scenarios (e.g., occlusions, invisible keypoints, and abnormal poses). Particularly, since mouse body is highly deformable, it is a big challenge to accurately locate different keypoints on the mouse body. In this paper, we propose a novel hourglass network based model, namely Graphical Model based Structured Context Enhancement Network (GM-SCENet) where two effective modules, i.e., Structured Context Mixer (SCM) and Cascaded Multi-Level Supervision module (CMLS) are designed. The SCM can adaptively learn and enhance the proposed structured context information of each mouse part by a novel graphical model with close consideration on the difference between body parts. Then, the CMLS module is designed to jointly train the proposed SCM and the hourglass network by generating multi-level information, which increases the robustness of the whole network. Based on the multi-level predictions from the SCM and the CMLS module, we also propose an inference method to enhance the localization results. Finally, we evaluate our proposed approach against several baselines on our Parkinson's Disease Mouse Behaviour (PDMB) and the standard DeepLabCut Mouse Pose datasets, where the results show that our method can achieve better or competitive performance against the other state-of-theart approaches.

Smoothness-based Consistency Learning for Macaque Pose Estimation

“…To quantitatively measure animal behavior, recent studies [1-4, 6, 12, 13, 28, 29] have payed increasing attention to the field of animal pose estimation. Zhou et al [29] introduce a new challenging mouse dataset PDMB for mouse pose estimation and a novel Graphical Model based Structured Context Enhancement Network (GM-SCENet) to model mouse behaviour. Mu et al [12] generate an animal dataset with 10+ different animal CAD models and proposed a consistency-constrained semi-supervised learning framework (CC-SSL) to train synthetic and real dataset jointly.…”