A Multimodal Fusion Approach for Human Activity Recognition

Koutrintzes, Dimitrios; Spyrou, Evaggelos; Mathe, Eirini; Mylonas, Phivos

doi:10.1142/s0129065723500028

Cited by 6 publications

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We relied on our previous work [ 34 ], where we presented an image representation of 3D skeletal motion. Moreover, we used the same CNN architecture for classification as in [ 36 ]. Note that the goal of the following experiments was not to provide state-of-the-art results in human activity recognition, but to be used as a baseline so as to verify the effectiveness of our approach.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Human Activity Recognition in the Presence of Occlusion

Vernikos

Spyropoulos

Spyrou

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

The presence of occlusion in human activity recognition (HAR) tasks hinders the performance of recognition algorithms, as it is responsible for the loss of crucial motion data. Although it is intuitive that it may occur in almost any real-life environment, it is often underestimated in most research works, which tend to rely on datasets that have been collected under ideal conditions, i.e., without any occlusion. In this work, we present an approach that aimed to deal with occlusion in an HAR task. We relied on previous work on HAR and artificially created occluded data samples, assuming that occlusion may prevent the recognition of one or two body parts. The HAR approach we used is based on a Convolutional Neural Network (CNN) that has been trained using 2D representations of 3D skeletal motion. We considered cases in which the network was trained with and without occluded samples and evaluated our approach in single-view, cross-view, and cross-subject cases and using two large scale human motion datasets. Our experimental results indicate that the proposed training strategy is able to provide a significant boost of performance in the presence of occlusion.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Human Activity Recognition in the Presence of Occlusion

Vernikos

Spyropoulos

Spyrou

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of the fields that has significantly benefited from the aforementioned advances is the one of Human Activity Recognition (HAR) [ 1 , 2 , 3 ]. HAR approaches may be applied in several tasks, such as video surveillance [ 4 ], health/elderly care [ 5 ], human–computer interactions and/or automation [ 6 , 7 ], sports analysis/training [ 8 ], behavior analysis [ 9 ] etc.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Modal Egocentric Activity Recognition Approach towards Video Domain Generalization

Papadakis,

Spyrou

2024

Sensors

Self Cite

View full text Add to dashboard Cite

Egocentric activity recognition is a prominent computer vision task that is based on the use of wearable cameras. Since egocentric videos are captured through the perspective of the person wearing the camera, her/his body motions severely complicate the video content, imposing several challenges. In this work we propose a novel approach for domain-generalized egocentric human activity recognition. Typical approaches use a large amount of training data, aiming to cover all possible variants of each action. Moreover, several recent approaches have attempted to handle discrepancies between domains with a variety of costly and mostly unsupervised domain adaptation methods. In our approach we show that through simple manipulation of available source domain data and with minor involvement from the target domain, we are able to produce robust models, able to adequately predict human activity in egocentric video sequences. To this end, we introduce a novel three-stream deep neural network architecture combining elements of vision transformers and residual neural networks which are trained using multi-modal data. We evaluate the proposed approach using a challenging, egocentric video dataset and demonstrate its superiority over recent, state-of-the-art research works.

show abstract