MEBOW: Monocular Estimation of Body Orientation in the Wild

Abstract

Body orientation estimation provides crucial visual cues in many applications, including robotics and autonomous driving. It is particularly desirable when 3-D pose estimation is difficult to infer due to poor image resolution, occlusion, or indistinguishable body parts. We present COCO-MEBOW (Monocular Estimation of Body Orientation in the Wild), a new large-scale dataset for orientation estimation from a single in-the-wild image. The body-orientation labels for around 130K human bodies within 55K images from the COCO dataset have been collected using an efficient and high-precision annotation pipeline. We also validated the benefits of the dataset. First, we show that our dataset can substantially improve the performance and the robustness of a human body orientation estimation model, the development of which was previously limited by the scale and diversity of the available training data. Additionally, we present a novel triple-source solution for 3-D human pose estimation, where 3-D pose labels, 2-D pose labels, and our body-orientation labels are all used in joint training. Our model significantly outperforms state-of-the-art dual-source solutions for monocular 3-D human pose estimation, where training only uses 3-D pose labels and 2-D pose labels. This substantiates an important advantage of MEBOW for 3-D human pose estimation, which is particularly appealing because the per-instance labeling cost for body orientations is far less than that for 3-D poses. The work demonstrates high potential of MEBOW in addressing real-world challenges involving understanding human behaviors. Further information of this work is available at https://chenyanwu.github.io/MEBOW/ .

Bodily expressed emotion understanding through integrating Laban movement analysis

“…The rest of the BoME samples are exclusive to LMA labels, whereas the remaining samples in the BoLD set contain only emotion labels. Consequently, drawing inspiration from the work of Wu et al., 43 we adopted a weakly supervised training methodology, enabling us to effectively leverage data that lack either emotion or LMA labels.…”

Economical Quaternion Extraction from a Human Skeletal Pose Estimate using 2-D Cameras

“…While the proposed system is straightforward, it lacks novelty and does not compare to state-of-the-art methods. The literature review is also inadequate in terms of recent works on body orientation [9], [10], and it is unclear which model in MediaPipe is used. Moreover, the proposed algorithm assumes 3D joint positions are given by the estimator, which may not be accurately extracted from 2D images.…”

“…To expand on the review, recent works on body orientation estimation have been focused on improving the accuracy and robustness of the estimation algorithms. For example, the MeBoW system proposed by We et al [9] uses a combination of 2D pose estimation and monocular depth estimation to estimate 3D body poses and orientations accurately. Zhou et al [10] proposed a joint regression method that uses 2D joint positions and depth maps to estimate body orientation accurately.…”

Automatic Detection of Gaze and Body Orientation in Elementary School Classrooms

“…Although there are a number of algorithms to determine the orientation of people using only one camera ( Moreno-Noguer, 2016 ; de Paiva et al, 2020 ; Wu et al, 2020 ), they have several limitations for use in the classroom. They are focused on pedestrian body orientation and other situations that are of interest to autonomous vehicles.…”

“…They are focused on pedestrian body orientation and other situations that are of interest to autonomous vehicles. For example, they estimate the orientation of the body as an angle in the ground plane ( Wu et al, 2020 ). Furthermore, they are not trained using elementary school classroom databases, where most of the students are seated and with strong occlusion between them and with classroom furniture.…”