A Fine-to-Coarse Convolutional Neural Network for 3D Human Action Recognition

Le, Thao Minh; Inoue, Nakamasa; Shinoda, Koichi

doi:10.48550/arxiv.1805.11790

Cited by 4 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The best performance was recorded by the CNN models. Although RNNs and HNNs employ recurrent layers that are specifically designed for processing sequential data, the results were not too surprising to our team for two key reasons: (1) the employed dataset is fairly small, consisting of less than 200 repetitions per exercises, and (2) a growing body of work report of improved performance by CNNs on time-series and movement data [43], [44]. More specifically, recurrent networks utilize a larger number of parameters, thus they are more prone to overfitting on smaller datasets.…”

Section: Discussionmentioning

confidence: 94%

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises

Liao

Vakanski

Xian

2020

IEEE Trans. Neural Syst. Rehabil. Eng.

141

120

View full text Add to dashboard Cite

Computer-aided assessment of physical rehabilitation entails evaluation of patient performance in completing prescribed rehabilitation exercises, by processing movement data captured with a sensory system. Despite the essential role of rehabilitation assessment toward improved rehabilitation outcomes and reduced healthcare costs, existing approaches for computeraided monitoring and evaluation of patient performance lack versatility, robustness, and practical relevance. In this paper, we propose a deep learning-based framework for automated assessment of the quality of physical rehabilitation exercises. The main components of the framework are metrics for quantifying movement performance, scoring functions for mapping the performance metrics into numerical scores of movement quality, and deep neural network models for regressing quality scores of input movements via supervised learning. A performance metric based on the log-likelihood of a Gaussian mixture model used for encoding low-dimensional data representation obtained with a deep autoencoder network, is proposed in the paper. Multiple deep network architectures are repurposed for the task in hand and are validated by using a dataset of rehabilitation exercises. To the best of our knowledge, this is the first work that implements deep neural networks for the assessment of rehabilitation performance.

show abstract

Section: Discussionmentioning

confidence: 94%

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises

Liao

Vakanski

Xian

2020

IEEE Trans. Neural Syst. Rehabil. Eng.

141

120

View full text Add to dashboard Cite

show abstract

“…There is also a large volume of work on two-person interaction classification from videos (e.g., [ 31 ]) and skeletal data (e.g., [ 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ]). Some of these models incorporate temporal [ 31 , 37 ], spatial and temporal [ 34 ], or multilayer feature [ 35 ] attention mechanisms.…”

Section: Related Workmentioning

confidence: 99%

Attention-Based Variational Autoencoder Models for Human–Human Interaction Recognition via Generation

Banerjee,

Baruah

2024

Sensors

View full text Add to dashboard Cite

The remarkable human ability to predict others’ intent during physical interactions develops at a very early age and is crucial for development. Intent prediction, defined as the simultaneous recognition and generation of human–human interactions, has many applications such as in assistive robotics, human–robot interaction, video and robotic surveillance, and autonomous driving. However, models for solving the problem are scarce. This paper proposes two attention-based agent models to predict the intent of interacting 3D skeletons by sampling them via a sequence of glimpses. The novelty of these agent models is that they are inherently multimodal, consisting of perceptual and proprioceptive pathways. The action (attention) is driven by the agent’s generation error, and not by reinforcement. At each sampling instant, the agent completes the partially observed skeletal motion and infers the interaction class. It learns where and what to sample by minimizing the generation and classification errors. Extensive evaluation of our models is carried out on benchmark datasets and in comparison to a state-of-the-art model for intent prediction, which reveals that classification and generation accuracies of one of the proposed models are comparable to those of the state of the art even though our model contains fewer trainable parameters. The insights gained from our model designs can inform the development of efficient agents, the future of artificial intelligence (AI).

show abstract

“…We compare our model with other methods in Table 3, including LRCN [27], 3D-ConvNet [28], Two-StreamI3D [23], Two-Stream-CBAM-I3D [24], and AFSD [29]. The metrics used are accuracy, precision, recall, and a weighted composite evaluation index of the three for the experimental evaluation of action recognition.…”

Section: Experiments Of Action Recognitionmentioning

confidence: 99%

Video Process Mining and Model Matching for Intelligent Development: Conformance Checking

Chen

Zou

Cao

et al. 2023

Sensors

View full text Add to dashboard Cite

Traditional business process-extraction models mainly rely on structured data such as logs, which are difficult to apply to unstructured data such as images and videos, making it impossible to perform process extractions in many data scenarios. Moreover, the generated process model lacks analysis consistency of the process model, resulting in a single understanding of the process model. To solve these two problems, a method of extracting process models from videos and analyzing the consistency of process models is proposed. Video data are widely used to capture the actual performance of business operations and are key sources of business data. Video data preprocessing, action placement and recognition, predetermined models, and conformance verification are all included in a method for extracting a process model from videos and analyzing the consistency between the process model and the predefined model. Finally, the similarity was calculated using graph edit distances and adjacency relationships (GED_NAR). The experimental results showed that the process model mined from the video was better in line with how the business was actually carried out than the process model derived from the noisy process logs.

show abstract

A Fine-to-Coarse Convolutional Neural Network for 3D Human Action Recognition

Cited by 4 publications

References 27 publications

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises

Attention-Based Variational Autoencoder Models for Human–Human Interaction Recognition via Generation

Video Process Mining and Model Matching for Intelligent Development: Conformance Checking

Contact Info

Product

Resources

About