Marker-Less Monitoring Protocol to Analyze Biomechanical Joint Metrics During Pedaling

Abstract

Marker-less systems are becoming popular to detect a human skeleton in an image automatically. However, these systems have difficulties in tracking points when part of the body is hidden, or there is an artifact that does not belong to the subject (e.g., a bicycle). We present a low-cost tracking system combined with economic force-measurement sensors that allows the calculation of individual joint moments and powers affordable for anybody. The system integrates OpenPose (deep-learning based C++ library to detect human skeletons in an image) in a system of two webcams, to record videos of a cyclist, and seven resistive sensors to measure forces at the pedals and the saddle. OpenPose identifies the skeleton candidate using a convolution neural network. A corrective algorithm was written to automatically detect the hip, knee, ankle, metatarsal and heel points from webcam-recorded motions, which overcomes the limitations of the marker-less system. Then, with the information of external forces, an inverse dynamics analysis is applied in OpenSim to calculate the joint moments and powers at the hip, knee, and ankle joints. The results show that the obtained moments have similar shapes and trends compared to the literature values. Therefore, this represents a low-cost method that could be used to estimate relevant joint kinematics and dynamics, and consequently follow up or improve cycling training plans.

Applications and limitations of current markerless motion capture methods for clinical gait biomechanics

“…However, self-occlusion errors are a major issue, often causing joint center locations to be missing for one or more frames and contribute to instances where the opposite limb is incorrectly detected ( e.g. , right knee labelled as the left knee) ( Serrancolí et al, 2020 ; Stenum, Rossi & Roemmich, 2021 ). Similar to marker-based methods, obtaining biomechanically relevant 2D planar joint angles requires an assumption that the camera is perfectly aligned with frontal or sagittal plane movements ( Stenum, Rossi & Roemmich, 2021 ).…”

“…While not strictly monocular, Serrancolí et al (2020) and Stenum, Rossi & Roemmich (2021) used two video cameras (25–60 Hz), placed on either side of a person, to extract information of the side closest to each camera and negate occlusion errors during walking over-ground or cycling on an ergometer. During walking, temporal differences were on average within 1 frame and spatial differences were less than one cm, although maximum differences were as high as 20 cm ( Stenum, Rossi & Roemmich, 2021 ).…”

“…Both studies also required additional manual input to fix incorrectly detected joints ( e.g. , right knee labelled as the left knee) ( Serrancolí et al, 2020 ; Stenum, Rossi & Roemmich, 2021 ). Therefore, some 2D monocular methods may obtain temporospatial (DeepLabCut and OpenPose) and planar 2D joint angles (DeepLabCut) with accuracy similar to marker-based motion capture ( Miranda et al, 2013 ), but this has only been examined for the side of the body closest to the camera.…”

Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 1: Robustness

“…Both works trained their own model with DeepLabCut. Serrancoli et al [ 30 ] fused OpenPose and force sensors to retrieve joint dynamics in a pedaling task.…”

Validity of Neural Networks to Determine Body Position on the Bicycle

“…Nevertheless, whenever markers are properly attached to bony landmarks they are considered a gold standard method. The use of marker-less methods to extract joint centres from video has been attempted in several studies (Grigg et al, 2018;Needham et al, 2017;Ong et al, 2017;Serrancolí et al, 2020). Ong et al (2017) observed differences of <1° for various joint angles using a marker-less tracking system during walking and jogging, demonstrating promising outcomes.…”