Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation

Abstract

Achieving robust multi-person 2D body landmark localization and pose estimation is essential for human behavior and interaction understanding as encountered for instance in HRI settings. Accurate methods have been proposed recently, but they usually rely on rather deep Convolutional Neural Network (CNN) architecture, thus requiring large computational and training resources. In this paper, we investigate different architectures and methodologies to address these issues and achieve fast and accurate multi-person 2D pose estimation. To foster speed, we propose to work with depth images, whose structure contains sufficient information about body landmarks while being simpler than textured color images and thus potentially requiring less complex CNNs for processing. In this context, we make the following contributions. i) we study several CNN architecture designs combining pose machines relying on the cascade of detectors concept with lightweight and efficient CNN structures; ii) to address the need for large training datasets with high variability, we rely on semi-synthetic data combining multi-person synthetic depth data with real sensor backgrounds; iii) we explore domain adaptation techniques to address the performance gap introduced by testing on real depth images; iv) to increase the accuracy of our fast lightweight CNN models, we investigate knowledge distillation at several architecture levels which effectively enhance performance. Experiments and results on synthetic and real data highlight the impact of our design choices, providing insights into methods addressing standard issues normally faced in practical applications, and resulting in architectures effectively matching our goal in both performance and speed.Index Terms-Human Pose Estimation, Convolutional Neural Networks, Machine Learning.CNN-based human pose estimation methods traditionally use a deep architecture pretrained on a large scale image recognition dataset. This design choice might unnecessary bring high computational burden. In this paper, inspired by efficient network structures such as those encountered in ResNets [7], MobileNets [8] and SqueezeNets [9], we introduce novel lightweight network architectures that match our real-time

Research on Multicamera Photography Image Art in BERT Motion Based on Deep Learning Mode

“…From the perspective of the development process of image text description, it can be divided into three stages: template-based image text description method; retrieval-based image text description method; deep learning-based image text description method [ 1 ]. Before the deep learning method was proposed, most of the image description methods used template-based and retrieval-based methods.…”

Residual Pose: A Decoupled Approach for Depth-based 3D Human Pose Estimation

“…Three architectures are considered. The two firsts are the efficient pose machines based on residual modules (RPM) and the one based on MobileNets (MPM) introduced in [14]. These are lightweight CNNs that refine predictions with a series of prediction stages and are designed for efficient 2D pose estimation with real-time performance, see Fig.…”

“…2D CNN architectures and training. We keep the performance-efficiency trade-off reported in [14] and experiment with RPM with 2 stages and MPM with 4 stages. We configure the Hourglass architecture (HG) to 2 stages it was shown that performance saturates at this point [16].…”

A system for the generation of in-car human body pose datasets

“…Chen et al [7] created an automated toolchain to synthesize RGB images from 3D poses. Regarding depth features, Martinez et al [21] combined multi-person synthetic depth data with real sensor backgrounds.…”