Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 1: Robustness

Abstract

Being able to capture relevant information about elite athletes’ movement “in the wild” is challenging, especially because reference marker-based approaches hinder natural movement and are highly sensitive to environmental conditions. We propose Pose2Sim, a markerless kinematics workflow that uses OpenPose 2D pose detections from multiple views as inputs, identifies the person of interest, robustly triangulates joint coordinates from calibrated cameras, and feeds those to a 3D inverse kinematic full-body OpenSim model in order to compute biomechanically congruent joint angles. We assessed the robustness of this workflow when facing simulated challenging conditions: (Im) degrades image quality (11-pixel Gaussian blur and 0.5 gamma compression); (4c) uses few cameras (4 vs. 8); and (Cal) introduces calibration errors (1 cm vs. perfect calibration). Three physical activities were investigated: walking, running, and cycling. When averaged over all joint angles, stride-to-stride standard deviations lay between 1.7° and 3.2° for all conditions and tasks, and mean absolute errors (compared to the reference condition—Ref) ranged between 0.35° and 1.6°. For walking, errors in the sagittal plane were: 1.5°, 0.90°, 0.19° for (Im), (4c), and (Cal), respectively. In conclusion, Pose2Sim provides a simple and robust markerless kinematics analysis from a network of calibrated cameras.

Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 2: Accuracy

“…We previously proposed Pose2Sim [ 25 ], an open-source markerless kinematics workflow using a network of calibrated RGB cameras, bridging OpenPose [ 13 ] to OpenSim. OpenSim is open-source 3D biomechanical analysis software that uses a multi-body optimization approach to solve inverse kinematics [ 26 , 27 ].…”

“…OpenSim is open-source 3D biomechanical analysis software that uses a multi-body optimization approach to solve inverse kinematics [ 26 , 27 ]. Our previous study [ 25 ] showed that Pose2Sim was robust to dark and blurry images (0.5 gamma compression and 5.5 cm Gaussian blur), to 1 cm random calibration errors, and to using as few as four cameras. Because Needham et al showed that the quality of markerless results were task specific [ 28 ], we examined walking, running, and cycling.…”

A SWOT Analysis of Portable and Low-Cost Markerless Motion Capture Systems to Assess Lower-Limb Musculoskeletal Kinematics in Sport

“…In such studies, markerless motion capture has shown great promise. Specifically, focusing on the ability to detect lower extremity movement, multiple studies have indicated that markerless motion capture can efficiently capture spatiotemporal joint kinematic variables (Clark et al, 2013;Sandau et al, 2014;Mentiplay et al, 2015;Rocha et al, 2018) with moderate-to-high agreement during tasks such as a single leg squat (Perrott et al, 2017;Kotsifaki et al, 2018;Tipton et al, 2019), vertical jump (Drazan et al, 2021), countermovement jump (Kotsifaki et al, 2018), stair climbing (Ogawa et al, 2017), walking (Ceseracciu et al, 2014;Sandau et al, 2014;Kanko et al, 2021;Pagnon et al, 2021;Stenum et al, 2021;Takeda et al, 2021;Vafadar et al, 2021), running (Corazza et al, 2006;Macpherson et al, 2016;Pagnon et al, 2021), gymnastics tasks (Corazza et al, 2006(Corazza et al, , 2010Mündermann et al, 2007), and clinical evaluations (Eltoukhy et al, 2017;Mauntel et al, 2021). To date, the highest accuracy with markerless motion capture has been achieved when fitting a prior articulated model to a 3D surface visual hull reconstruction using matching algorithms (Corazza et al, 2006(Corazza et al, , 2007(Corazza et al, , 2008(Corazza et al, , 2010Mündermann et al, 2006bMündermann et al, , 2007.…”

“…To date, the highest accuracy with markerless motion capture has been achieved when fitting a prior articulated model to a 3D surface visual hull reconstruction using matching algorithms (Corazza et al, 2006(Corazza et al, , 2007(Corazza et al, , 2008(Corazza et al, , 2010Mündermann et al, 2006bMündermann et al, , 2007. More recently, however, the application of deep learning algorithms, keypoint detection approaches for biomechanical assessment are beginning to show similar or greater accuracies and illustrate significant promise for the future of markerless motion capture in the sports medicine domain (Drazan et al, 2021;Kanko et al, 2021;Needham et al, 2021;Pagnon et al, 2021;Stenum et al, 2021;Vafadar et al, 2021). It is important to note that the majority of validation studies utilizing markerless motion capture to assess joint kinematics evaluate relatively slow movements such as walking, or single plane motions such as the sagittal plane during jumping.…”

“…Unfortunately, the research into the development and improvement of these algorithms is outside the scope for most biomechanics research and, thus, the applicability of several of these algorithms for biomechanical use is still widely unknown (Mündermann et al, 2006a;Colyer et al, 2018). That being said, more recently researchers have begun to evaluate the more popular human pose estimation algorithms (e.g., OpenPose, DeepLabCut, AlphaPose) for their accuracy when applying them to the human movement sciences domain (Drazan et al, 2021;Needham et al, 2021;Pagnon et al, 2021;Stenum et al, 2021;Takeda et al, 2021) and perhaps with the growing popularity of applying markerless motion capture to the sports medicine domain, there will be a push for more biomechanically accurate algorithms. The benefit of using machine learningbased algorithms for estimating human movement is that these algorithms can be continually refined.…”