Continuous Human Action Recognition for Human-machine Interaction: A Review

Abstract

With advances in data-driven machine learning research, a wide variety of prediction models have been proposed to capture spatio-temporal features for the analysis of video streams. Recognising actions and detecting action transitions within an input video are challenging but necessary tasks for applications that require real-time human-machine interaction. By reviewing a large body of recent related work in the literature, we thoroughly analyse, explain and compare action segmentation methods and provide details on the feature extraction and learning strategies that are used on most state-of-the-art methods. We cover the impact of the performance of object detection and tracking techniques on human action segmentation methodologies. We investigate the application of such models to real-world scenarios and discuss several limitations and key research directions towards improving interpretability, generalisation, optimisation, and deployment.

CatStep: Automated Cataract Surgical Phase Classification and Boundary Segmentation Leveraging Inflated 3D-Convolutional Neural Network Architectures and BigCat

“…The Over-Segmentation Score measures the extent of overlap between GT and predicted segments. 16 This score is a function of the predicted segment with a maximum intersection over union for a given GT segment and is given by: where G = {G 0 … G i … G N } is the sequence of GT phases, and P = {P 0 … P j … P N }is the set of phase predictions. The Over-Segmentation Score lies within [0, 1] and a higher value indicates better performance.…”

“…The SES measures how well a model predicts the ordering of phases independent of slight time offsets. 16 , 18 Specifically, the SES allows for the evaluation of misclassifications, insertions, and deletions in phase predictions. To compute the SES, an edit distance is first calculated by identifying the minimum number of substitutions, deletions, and/or insertions required to transform the sequence of predicted phases ( P ) into the sequence of GT phases ( G ) using the Wagner–Fischer algorithm.…”

Adaptive Kalman Filter for Real-Time Visual Object Tracking Based on Autocovariance Least Square Estimation

“…Visual object tracking (VOT) is a fundamental task in computer vision and has extensive applications including autonomous vehicles [1], video surveillance [2], robot vision [3], and human-computer interaction [4]. Specifically, in autonomous vehicle systems and robotics, robust and efficient VOT algorithms that identify and track nearby vehicles and pedestrians are essential for real-time navigation, obstacle avoidance, and environment perception, ensuring safe and efficient operation in dynamic scenarios.…”