LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

Abstract

In this paper, we place the atomic action detection problem into a Long-Short Term Context (LSTC) to analyze how the temporal reliance among video signals affect the action detection results. To do this, we decompose the action recognition pipeline into shortterm and long-term reliance, in terms of the hypothesis that the two kinds of context are conditionally independent given the objective action instance. Within our design, a local aggregation branch is utilized to gather dense and informative short-term cues, while a high order long-term inference branch is designed to reason the objective action class from high-order interaction between actor and other person or person pairs. Both branches independently predict the context-specific actions and the results are merged in the end. We demonstrate that both temporal grains are beneficial to atomic action recognition. On the mainstream benchmarks of atomic action detection, our design can bring significant performance gain from the existing state-of-the-art pipeline. CCS CONCEPTS• Computing methodologies → Activity recognition and understanding.

MSTA-SlowFast: A Student Behavior Detector for Classroom Environments

“…In place of the double-branch approach, Christoph [19] presented an extended 3D convolutional network (X3D), which gradually modifies the model's width parameter to require less computational work while producing superior results. Li et al [20] analyzed the effect of time dependence on behavior detection by placing the behavior detection in a Long-Short Term Context (LSTC).…”

Human‐centered attention‐aware networks for action recognition

“…Table 1 shows the comparison results between the proposed methods and the SOTA methods SlowFast R‐50 and SlowFast R‐101‐NL, 41 MviT‐B‐24, 46 AVSF, 44 LSTC, 47 X3D‐XL 45 on AVA2.2 data set. It is observed that the proposed method outperforms other methods, with the original ACAR algorithm being 30.5% and the mAP of the proposed network being increased to 30.7%.…”

“…Comparison Results of the overall performance of SlowFast, R‐50, 41 SlowFast, R‐101‐NL, 41 M‐viT‐B‐24, 46 AVSF, 44 LSTC, 47 X3D‐XL 45 on the AVA2.2 data set. [Color figure can be viewed at wileyonlinelibrary.com]…”

“…LSTC,47 X3D-XL45 on the AVA2.2 data set. [Color figure can be viewed at wileyonlinelibrary.com] T A B L E 2 Comparison of the proposed method with ACAR on AVA2.2 data set…”