BABEL: Bodies, Action and Behavior with English Labels

Punnakkal, Abhinanda R.; Chandrasekaran, Arjun; Athanasiou, Nikos; Quiros-Ramirez, Alejandra; Black, Michael J.

doi:10.1109/cvpr46437.2021.00078

Cited by 88 publications

(44 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The recent BABEL dataset [42] contains dense, framelevel action labels for mocap sequences from AMASS [35]. Actions are labeled post-hoc by human annotators.…”

Section: Human Labelsmentioning

confidence: 99%

“…PKU-MMD [32] consists of 51 simple actions such as 'putting on glasses', 'shake hands', which were specifically performed by actors. In contrast, the recent BABEL dataset [42] consists of mocap sequences from the large AMASS data archive [35], a consolidation of many mocap datasets. BABEL is more complex and has a long-tailed distribution of actions.…”

Section: Related Workmentioning

confidence: 99%

“…We propose the recently released BABEL dataset [42] as a new 3D-TAL benchmark. BABEL is diverse, containing more than 250 actions whose frequency follows a long-tailed distribution.…”

Section: Babel-tal-60 (Bt60)mentioning

confidence: 99%

“…The BABEL dataset [42] consists of more than 250 actions whose frequency follows a long-tailed distribution. To ensure that the learning methods have access to sufficient data, the BABEL action recognition benchmark only considers a subset of actions for the task.…”

Section: A Datasetmentioning

confidence: 99%

“…We create the BABEL-TAL-60 (BT60) benchmark for 3D-TAL the 60-class action recognition subset of BABEL presented by Punnakkal et al [42]. The action recognition task involves predicting the action class of the given 'trimmed' span of movement.…”

Section: A Datasetmentioning

confidence: 99%

See 4 more Smart Citations

LocATe: End-to-end Localization of Actions in 3D with Transformers

Sun¹,

Zhou²,

Black³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Understanding a person's behavior from their 3D motion is a fundamental problem in computer vision with many applications. An important component of this problem is 3D Temporal Action Localization (3D-TAL), which involves recognizing what actions a person is performing, and when. State-of-the-art 3D-TAL methods employ a two-stage approach in which the action span detection task and the action recognition task are implemented as a cascade. This approach, however, limits the possibility of error-correction. In contrast, we propose LocATe, an end-to-end approach that jointly localizes and recognizes actions in a 3D sequence. Further, unlike existing autoregressive models that focus on modeling the local context in a sequence, LocATe's transformer model is capable of capturing long-term correlations between actions in a sequence. Unlike transformer-based object-detection and classification models which consider image or patch features as input, the input in 3D-TAL is a long sequence of highly correlated frames. To handle the high-dimensional input, we implement an effective input representation, and overcome the diffuse attention across long time horizons by introducing sparse attention in the model. LocATe outperforms previous approaches on the existing PKU-MMD 3D-TAL benchmark (mAP = 93.2%). Finally, we argue that benchmark datasets are most useful where there is clear room for performance improvement. To that end, we introduce a new, challenging, and more realistic benchmark dataset, BABEL-TAL-20 (BT20), where the performance of state-of-the-art methods is significantly worse. The dataset and code for the method will be available for research purposes.

show abstract

“…The recent BABEL dataset [42] contains dense, framelevel action labels for mocap sequences from AMASS [35]. Actions are labeled post-hoc by human annotators.…”

Section: Human Labelsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…We propose the recently released BABEL dataset [42] as a new 3D-TAL benchmark. BABEL is diverse, containing more than 250 actions whose frequency follows a long-tailed distribution.…”

Section: Babel-tal-60 (Bt60)mentioning

confidence: 99%

Section: A Datasetmentioning

confidence: 99%

Section: A Datasetmentioning

confidence: 99%

See 3 more Smart Citations