Semi-supervised Human Pose Estimation in Art-historical Images

Abstract

Gesture as 'language' of non-verbal communication has been theoretically established since the 17th century. However, its relevance for the visual arts has been expressed only sporadically. This may be primarily due to the sheer overwhelming amount of data that traditionally had to be processed by hand. With the steady progress of digitization, though, a growing number of historical artifacts have been indexed and made available to the public, creating a need for automatic retrieval of art-historical motifs with similar body constellations or poses. Since the domain of art differs significantly from existing real-world data sets for human pose estimation due to its style variance, this presents new challenges. In this paper, we propose a novel approach to estimate human poses in art-historical images. In contrast to previous work that attempts to bridge the domain gap with pre-trained models or through style transfer, we suggest semi-supervised learning for both object and keypoint detection. Furthermore, we introduce a novel domain-specific art data set that includes both bounding box and keypoint annotations of human figures. Our approach achieves significantly better results than methods that use pre-trained models or style transfer.

MVCRNet: A Semi-Supervised Multi-View Framework for Robust Animal Pose Estimation with Minimal Labeled Data

“…Animal pose estimation has lagged behind human pose estimation due to the lack of large-scale data sets analogous to COCO [22]. Despite significant achievements in semi-supervised human pose estimation [23,24,37], differences between animals and humans in posture, appearance, and body structure have limited the effectiveness of certain semi-supervised frameworks when applied to animals. Notably, the semi-supervised human pose estimation framework DualPose [25] uses a dual network structure to guide relatively accurate predictions to another prediction, reducing the differences between teacher and student networks and demonstrating excellent performance.…”