FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction

Gordon, Brian; Raab, Sigal; Guy, Azov,; Giryes, Raja; Cohen–Or, Daniel

doi:10.1007/978-3-031-19827-4_11

Cited by 18 publications

(8 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results show that our method outperforms other approaches in both scenarios. In the second scenario, our MPJPE is 20.2mm (30.8%) lower than (Gordon et al 2022), proving that the model benefits from more training data categories. In the first scenario, our method is also highly competitive, with MPJPE decreasing by 2.3mm (3.5%) compared to (Gordon et al 2022).…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 84%

“…In the second scenario, our MPJPE is 20.2mm (30.8%) lower than (Gordon et al 2022), proving that the model benefits from more training data categories. In the first scenario, our method is also highly competitive, with MPJPE decreasing by 2.3mm (3.5%) compared to (Gordon et al 2022). These show efficacy of our method in handling challenging in-the-wild data.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 84%

“…However, they are difficult in adjusting to various scenes because they are too reliant on camera settings. When compared with multi-view 3D HPE methods without pre-providing camera parameters, we achieve superior results, with MPJPE decreasing by 2.6mm (8.6%) compared to (Gordon et al 2022). When given ground-truth 2D pose, as shown in Table 2, our model performance improves, with MPJPE 15.9mm (57.6%) and 3.6mm (23.5%) lower than Ours (CPN) and (Shuai, Wu, and Liu 2022), respectively.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 89%

See 2 more Smart Citations

Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation

Zhang,

Zhou,

et al. 2024

AAAI

View full text Add to dashboard Cite

Most Graph Convolutional Networks based 3D human pose estimation (HPE) methods were involved in single-view 3D HPE and utilized certain spatial graphs, existing key problems such as depth ambiguity, insufficient feature representation, or limited receptive fields. To address these issues, we propose a multi-view 3D HPE framework based on deep semantic graph transformer, which adaptively learns and fuses multi-view significant semantic features of human nodes to improve 3D HPE performance. First, we propose a deep semantic graph transformer encoder to enrich spatial feature information. It deeply mines the position, spatial structure, and skeletal edge knowledge of joints and dynamically learns their correlations. Then, we build a progressive multi-view spatial-temporal feature fusion framework to mitigate joint depth uncertainty. To enhance the pose spatial representation, deep spatial semantic feature are interacted and fused across different viewpoints during monocular feature extraction. Furthermore, long-time relevant temporal dependencies are modeled and spatial-temporal information from all viewpoints is fused to intermediately supervise the depth. Extensive experiments on three 3D HPE benchmarks show that our method achieves state-of-the-art results. It can effectively enhance pose features, mitigate depth ambiguity in single-view 3D HPE, and improve 3D HPE performance without providing camera parameters. Codes and models are available at https://github.com/z0911k/SGraFormer.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 84%

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 84%

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 89%

See 1 more Smart Citation

Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation

Zhang,

Zhou,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…21 Advancements in deep learning (DL) have improved the accuracy of video-based markerless 3D motion capture systems with single-view or multiview camera setups. 11,28,29,33 Recent data show that markerless solutions can provide satisfactory quality to be used in practice. 21,23 Our primary goal in this study was to develop an automated video analysis system that can function in such a complex and fast-paced environment.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

Identifying Anterior Cruciate Ligament Injuries Through Automated Video Analysis of In-Game Motion Patterns

Schulc,

Leite,

Csákvári

et al. 2024

Orthopaedic Journal of Sports Medicine

View full text Add to dashboard Cite

Background: Failure to diagnose anterior cruciate ligament (ACL) injury during a game can delay adequate treatment and increase the risk of further injuries. Artificial intelligence (AI) has the potential to be an accurate, cost-efficient, and readily available diagnostic tool for ACL injury in in-game situations. Purpose: To develop an automated video analysis system that uses AI to identify biomechanical patterns associated with ACL injury and to evaluate whether the system can enhance the ability of orthopaedic and sports medicine specialists to identify ACL injuries on video. Study Design: Descriptive laboratory study. Methods: A total of 91 ACL injury and 38 control movement scenes from online available match recordings were analyzed. The videos were processed to identify and track athletes and to estimate their 3-dimensional (3D) poses. Geometric features, including knee flexion, knee and hip abduction, and foot and hip rotation, were extracted from the athletes’ 3D poses. A recurrent neural network algorithm was trained to classify ACL injury, using these engineered features as its input. Analysis by 2 orthopaedic surgeons examined whether providing clinical experts with the reconstructed 3D poses and their derived signals could increase their diagnostic accuracy. Results: All AI models performed significantly better than chance. The best model, which used the long short-term memory network with engineered features, demonstrated decision interpretability and good performance (F1 score = 0.63 ± 0.01, area under the receiver operating characteristic curve = 0.88 ± 0.01). The analysis by the 2 orthopaedic surgeons demonstrated improved diagnostic accuracy for ACL injury recognition when provided with system data, resulting in a 0.08 increase in combined F1 scores. Conclusion: Our approach successfully reconstructed the 3D motion of athletes from a single-camera view and derived geometry-based biomechanical features from pose sequences. Our trained AI model was able to automatically detect ACL injuries with relatively good performance and prelabel and highlight regions of interest in video footage. Clinical Relevance: This study demonstrated the feasibility of using AI to automatically evaluate in-game video footage and identify dangerous motion patterns. Further research can explore the full potential of the biomechanical markers and use of the system by nonspecialists, potentially diminishing the rate of missed diagnosis and the detrimental outcomes that follow.

show abstract

3D Human Pose Estimation Based on Multi-feature Extraction

Shi

Zhang

et al. 2022

Artificial Intelligence

View full text Add to dashboard Cite

FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction

Cited by 18 publications

References 72 publications

Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation

Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation

Identifying Anterior Cruciate Ligament Injuries Through Automated Video Analysis of In-Game Motion Patterns

3D Human Pose Estimation Based on Multi-feature Extraction

Contact Info

Product

Resources

About