Multiview 3D Markerless Human Pose Estimation from OpenPose Skeletons

Abstract

Despite the fact that marker-based systems for human motion estimation provide very accurate tracking of the human body joints (at mm precision), these systems are often intrusive or even impossible to use depending on the circumstances, e.g. markers cannot be put on an athlete during competition. Instrumenting an athlete with the appropriate number of markers requires a lot of time and these markers may fall off during the analysis, which leads to incomplete data and requires new data capturing sessions and hence a waste of time and effort. Therefore, we present a novel multiview video-based markerless system that uses 2D joint detections per view (from OpenPose) to estimate their corresponding 3D positions while tackling the people association problem in the process to allow the tracking of multiple persons at the same time.Our proposed system can perform the tracking in real-time at 20-25 fps. Our results show a standard deviation between 9.6 and 23.7 mm for the lower body joints based on the raw measurements only. After filtering the data, the standard deviation drops to a range between 6.6 and 21.3 mm. Our proposed solution can be applied to a large number of applications, ranging from sports analysis to virtual classrooms where submillimeter precision is not necessarily required, but where the use of markers is impractical.

Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 1: Robustness

“…All stages of the workflow contributed to its robustness. First, we never had to deal with limb swapping, unlike other studies [ 47 , 48 ]. This may be due to our use of the experimental Body_25b OpenPose model, which is claimed to decrease the number of false positives [ 67 ].…”

“…Slembrouck at al. [ 48 ] went a step further and tackled the issue of limb swapping and multiple persons detection. In case of multiple persons detection, one needs to make sure they triangulate the person detected on one camera to the same person detected on the other ones.…”

“…This is a strong assumption, especially outdoors where people and athletes pass by and an operator is often present. Although it is starting to be addressed, this is still very debated [ 48 , 49 , 50 , 51 ].…”

Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person Detection

“…Then, an assembling method was proposed to select the correct 3D pose from the multiview 2D joints by joint semantic meanings. Slembrouck et al [ 32 ] presented a novel multiview video-based markerless system for 3D human pose estimation. This method converts 2D skeleton detections from multiple camera views into 3D skeletons.…”

Human Movement Science in The Wild: Can Current Deep-Learning Based Pose Estimation Free Us from The Lab?

“…The OpenPose package was selected here as it is easy to install and use as well as being one of the only systems to provide a foot detection at the time of writing. As such OpenPose is a popular system and has started to receive some attention in movement sciences research 10,26–28 . AlphaPose employs a top-down approach to multi-person pose estimation, first detecting individual people within an image before applying pose-estimation to detect key points of each detected person 24 .…”