Fine-grained action segmentation using the semi-supervised action GAN

Abstract

In this paper we address the problem of continuous fine-grained action segmentation, in which multiple actions are present in an unsegmented video stream. The challenge for this task lies in the need to represent the hierarchical nature of the actions and to detect the transitions between actions, allowing us to localise the actions within the video effectively. We propose a novel recurrent semi-supervised Generative Adversarial Network (GAN) model for continuous fine-grained human action segmentation. Temporal context information is captured via a novel Gated Context Extractor (GCE) module, composed of gated attention units, that directs the queued context information through the generator model, for enhanced action segmentation. The GAN is made to learn features in a semi-supervised manner, enabling the model to perform action classification jointly with the standard, unsupervised, GAN learning procedure.We perform extensive evaluations on different architectural variants to demonstrate the importance of the proposed network architecture, and show that it is capable of outperforming current state-of-the-art on three challenging datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities dataset.

Deep Learning for Medical Anomaly Detection -- A Survey

“…Hence, if the training mechanism can leverage information in unlabelled samples, it could be highly beneficial. The sub-field of semi-supervised learning addresses this situation and GANs have also demonstrated tremendous success in a semi-supervised setting [102] where the trained discriminator is adapted to perform the normal abnormal classification task, instead of real/fake validation [100]. However, we observe that deep medical anomaly detection methods rarely utilise semisupervised learning strategies.…”

“…Hence, like the autoencoder methods discussed for MRI anomaly detection Section II-C1, this is not a completely unsupervised model. Rather this architecture is semi-supervised, where both labelled and unlabelled examples are used for model training [102].…”

When AI meets store layout design: a review

“…Understanding and recognizing behaviors of a customer are based on the body gestures of the customer in relation to the shelves, the products and the trolley/basket he/she is using (Gammulle et al 2020). The analysis is based on recognizing the head orientation, eye gazing, and 2-dimensional (2D) and 3-dimensional (3D) pose estimation as illustrated in Fig.…”

CIES: Cloud-based Intelligent Evaluation Service for video homework using CNN-LSTM network

“…We also noticed that many novel algorithms on video processing has been proposed recently. On the domain of video segmentation, a new model based on Generative Adversarial Network has been proved to be more accuracy [19]. Ke et al propose Frame Segmentation Network, improving mAP (mean average precision) and IoU (Intersection over Union) simultaneously [20].…”