Online human action detection and anticipation in videos: A survey

Hu, Xuejiao; Dai, Jingzhao; Li, Ming; Peng, Chenglei; Li, Yang; Du, Sidan

doi:10.1016/j.neucom.2022.03.069

Cited by 19 publications

(8 citation statements)

References 139 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A segmentation procedure has two outputs: (1) the start and stop timestamps of the motion sequence and (2) what type of motion the sequence is (98)(99)(100). This requirement means that a sequence can potentially have multiple classes of motions, which is considered a more challenging problem than predicting the motion class of an already trimmed segment consisting of only one class.…”

Section: Movement Segmentationmentioning

confidence: 99%

“…Functional movement segmentation approaches, also known as human action detection in the computational literature (98)(99)(100), typically use supervised or unsupervised learning. Progress in functional movement segmentation algorithms for UE functional motions has benefited from the availability of labeled data sets.…”

Section: Functional Movement Segmentationmentioning

confidence: 99%

“…However, the only rehabilitation-focused UE functional motion data set ( 109 ) could not be found online. Zhang et al ( 99 ) and Hu et al ( 100 ) include state-of-the-art action detection performance measures for a variety of data sets, and most of the referenced data sets in Table 4 include benchmark performance results using supervised and unsupervised approaches.…”

Section: Movement Segmentationmentioning

confidence: 99%

See 2 more Smart Citations

Computer-assisted approaches for measuring, segmenting, and analyzing functional upper extremity movement: a narrative review of the current state, limitations, and future directions

Jackson

Duric

Engdahl

et al. 2023

Front. Rehabil. Sci.

View full text Add to dashboard Cite

The analysis of functional upper extremity (UE) movement kinematics has implications across domains such as rehabilitation and evaluating job-related skills. Using movement kinematics to quantify movement quality and skill is a promising area of research but is currently not being used widely due to issues associated with cost and the need for further methodological validation. Recent developments by computationally-oriented research communities have resulted in potentially useful methods for evaluating UE function that may make kinematic analyses easier to perform, generally more accessible, and provide more objective information about movement quality, the importance of which has been highlighted during the COVID-19 pandemic. This narrative review provides an interdisciplinary perspective on the current state of computer-assisted methods for analyzing UE kinematics with a specific focus on how to make kinematic analyses more accessible to domain experts. We find that a variety of methods exist to more easily measure and segment functional UE movement, with a subset of those methods being validated for specific applications. Future directions include developing more robust methods for measurement and segmentation, validating these methods in conjunction with proposed kinematic outcome measures, and studying how to integrate kinematic analyses into domain expert workflows in a way that improves outcomes.

show abstract

Section: Movement Segmentationmentioning

confidence: 99%

Section: Functional Movement Segmentationmentioning

confidence: 99%

Section: Movement Segmentationmentioning

confidence: 99%

See 1 more Smart Citation

Computer-assisted approaches for measuring, segmenting, and analyzing functional upper extremity movement: a narrative review of the current state, limitations, and future directions

Jackson

Duric

Engdahl

et al. 2023

Front. Rehabil. Sci.

View full text Add to dashboard Cite

show abstract

“…Online Action Detection (OAD) aims to detect whether an untrimmed human action video contains a target action and gives the category of the action (Hu et al 2022), so it is a classification task of all frames of video corresponding to one label (many-to-one). (Wang et al 2021) combines action prediction with online action detection, and applies Transformer architecture to improve model performance.…”

Section: Online Action Detectionmentioning

confidence: 99%

Streaming Video Temporal Action Segmentation In Real Time

Wang¹,

Li²,

Dong³

et al. 2022

Preprint

View full text Add to dashboard Cite

Temporal action segmentation (TAS) is a critical step toward long-term video understanding. Recent studies follow a pattern that builds models based on features instead of raw video picture information. However, we claim those models are trained complicatedly and limit application scenarios. It is hard for them to segment human actions of video in real time because they must work after the full video features are extracted. As the real-time action segmentation task is different from TAS task, we define it as streaming video real-time temporal action segmentation (SVTAS) task. In this paper, we propose a real-time end-to-end multi-modality model for SVTAS task. More specifically, under the circumstances that we cannot get any future information, we segment the current human action of streaming video chunk in real time. Furthermore, the model we propose combines the last steaming video chunk feature extracted by language model with the current image feature extracted by image model to improve the quantity of real-time temporal action segmentation. To the best of our knowledge, it is the first multi-modality real-time temporal action segmentation model. Under the same evaluation criteria as full video temporal action segmentation, our model segments human action in real time with less than 40% of state-of-the-art model computation and achieves 90% of the accuracy of the full video state-of-the-art model. Code is available at https://github.com/Thinksky5124/SVTAS.git.

show abstract

“…Temporal action detection (TAD) [1][2][3] is an offline action detection task that does not meet the requirements of real-time detection. Online action detection (OAD) [4] treats the TAD task as a per-frame classification challenge. However, accurately detecting the start position of actions within videos is more important than classifying each frame in time-sensitive scenarios.…”

Section: Introductionmentioning

confidence: 99%

Time‐attentive fusion network: An efficient model for online detection of action start

Hu,

Wang,

et al. 2024

IET Image Processing

Self Cite

View full text Add to dashboard Cite

Online detection of action start is a significant and challenging task that requires prompt identification of action start positions and corresponding categories within streaming videos. This task presents challenges due to data imbalance, similarity in boundary content, and real‐time detection requirements. Here, a novel Time‐Attentive Fusion Network is introduced to address the requirements of improved action detection accuracy and operational efficiency. The time‐attentive fusion module is proposed, which consists of long‐term memory attention and the fusion feature learning mechanism, to improve spatial‐temporal feature learning. The temporal memory attention mechanism captures more effective temporal dependencies by employing weighted linear attention. The fusion feature learning mechanism facilitates the incorporation of current moment action information with historical data, thus enhancing the representation. The proposed method exhibits linear complexity and parallelism, enabling rapid training and inference speed. This method is evaluated on two challenging datasets: THUMOS’14 and ActivityNet v1.3. The experimental results demonstrate that the proposed method significantly outperforms existing state‐of‐the‐art methods in terms of both detection accuracy and inference speed.

show abstract

Online human action detection and anticipation in videos: A survey

Cited by 19 publications

References 139 publications

Computer-assisted approaches for measuring, segmenting, and analyzing functional upper extremity movement: a narrative review of the current state, limitations, and future directions

Computer-assisted approaches for measuring, segmenting, and analyzing functional upper extremity movement: a narrative review of the current state, limitations, and future directions

Streaming Video Temporal Action Segmentation In Real Time

Time‐attentive fusion network: An efficient model for online detection of action start

Contact Info

Product

Resources

About