Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient Rehabilitation

Gavrilova, Marina L.; Ahmed, Ferdous; Bari, A. S. M. Hossain; Liu, Ruixuan; Liu, Tiantian; Maret, Yann; Sieu, Brandon; Sudhakar, Tanuja

doi:10.4018/978-1-5225-7525-2.ch007

Cited by 13 publications

(11 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-When is it more beneficial to use multi-modal gait/face emotion recognition system instead of face only/gait only emotion recognition? B Emergency response is another domain where accurate recognition of a human emotional state can be crucial for a success of a rescue operation or for mitigation of an immediate public risk [31], [58]. The areas for future investigations in this important domain are multiple: -Are there any other biometric modalities (such as EEG, heart rate etc.)…”

Section: Open Problemsmentioning

confidence: 99%

“…C Athlete performance and medical rehabilitation are areas where mental state plays almost as important role as physical fitness [32]- [34]. From that perspective, observing specific traits on body posture, joint movements and corresponding emotional expressions are paramount for successful outcome of an athlete's training program or a physical rehabilitation process [35], [36], [58]. Open problems arising from our research in relationship to those areas are: -Is there a correlation that can be detected by observing athlete training routine and their emotion state and can it be utilized for the development of a better training program?…”

Section: Open Problemsmentioning

confidence: 99%

See 1 more Smart Citation

Emotion Recognition From Body Movement

2020

Self Cite

View full text Add to dashboard Cite

Automatic emotion recognition from the analysis of body movement has tremendous potential to revolutionize virtual reality, robotics, behavior modeling, and biometric identity recognition domains. A computer system capable of recognizing human emotion from the body can also significantly change the way we interact with the computers. One of the significant challenges is to identify emotion-specific features from a vast number of descriptors of human body movements. In this paper, we introduce a novel two-layer feature selection framework for emotion classification from a comprehensive list of body movement features. We used the feature selection framework to accurately recognize five basic emotions: happiness, sadness, fear, anger, and neutral. In the first layer, a unique combination of Analysis of Variance (ANOVA) and Multivariate Analysis of Variance (MANOVA) was utilized to eliminate irrelevant features. In the second layer, a binary chromosome-based genetic algorithm was proposed to select a feature subset from the relevant list of features that maximizes the emotion recognition rate. Score and rank-level fusion were applied to further improve the accuracy of the system. The proposed system was validated on proprietary and public datasets, containing 30 subjects. Different action scenarios, such as walking and sitting actions, as well as an action-independent case, were considered. Based on the experimental results, the proposed emotion recognition system achieved a very high emotion recognition rate outperforming all of the state-of-theart methods. The proposed system achieved recognition accuracy of 90.0% during walking, 96.0% during sitting, and 86.66% in an action-independent scenario, demonstrating high accuracy and robustness of the developed method.

show abstract

Section: Open Problemsmentioning

confidence: 99%

Section: Open Problemsmentioning

confidence: 99%

Emotion Recognition From Body Movement

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Motion capture sensor-based gait emotion recognition from human motion is an emerging sub-domain of remote visual technologies. Remote visual technologies based on motion capture sensor and human motion have applications in a variety of domains, including smart home design, border security, unmanned aerial vehicles (UAVs), robotics, virtual reality, and gaming [ 1 ]. They have become even more essential in the current world of remote workplaces and virtual classrooms.…”

Section: Introductionmentioning

confidence: 99%

“…Consequently, Gait Emotion Recognition (GER) is defined as the process of determining emotions through a subject’s manner of walking [ 14 ]. GER can be beneficial in tuning and designing rehabilitation exercises [ 15 ], making emotionally intelligent robots [ 14 , 16 ], diagnosing and treating medical conditions [ 17 , 18 ], and identifying a person in distress during natural disasters [ 1 ]. The task requires body skeletal data (body joint coordinates over time) as the input, which can be collected from various types of motion capture sensors [ 19 , 20 ] or extracted from a video using pose-estimation methods [ 21 , 22 , 23 ].…”

Section: Introductionmentioning

confidence: 99%

Motion Capture Sensor-Based Emotion Recognition Using a Bi-Modular Sequential Neural Network

Bhatia

Bari

Hsu

et al. 2022

Sensors

View full text Add to dashboard Cite

Motion capture sensor-based gait emotion recognition is an emerging sub-domain of human emotion recognition. Its applications span a variety of fields including smart home design, border security, robotics, virtual reality, and gaming. In recent years, several deep learning-based approaches have been successful in solving the Gait Emotion Recognition (GER) problem. However, a vast majority of such methods rely on Deep Neural Networks (DNNs) with a significant number of model parameters, which lead to model overfitting as well as increased inference time. This paper contributes to the domain of knowledge by proposing a new lightweight bi-modular architecture with handcrafted features that is trained using a RMSprop optimizer and stratified data shuffling. The method is highly effective in correctly inferring human emotions from gait, achieving a micro-mean average precision of 0.97 on the Edinburgh Locomotive Mocap Dataset. It outperforms all recent deep-learning methods, while having the lowest inference time of 16.3 milliseconds per gait sample. This research study is beneficial to applications spanning various fields, such as emotionally aware assistive robotics, adaptive therapy and rehabilitation, and surveillance.

show abstract

“…Among all the aforementioned traits, gait is unobtrusive and easily collectable biometric, that can be observed without any hindrance to the subject's activity [2,3]. Gait recognition has been studied extensively in the domains of cybersecurity [4], computer graphics [5], assisted living [6], virtual reality [7], action recognition [8,9], emotion recognition [10], and medicine [11,12].…”

Section: Introductionmentioning

confidence: 99%

Residual connection-based graph convolutional neural networks for gait recognition

2021

Self Cite

View full text Add to dashboard Cite

The walking manner of a person, also known as gait, is a unique behavioral biometric trait. Existing methods for gait recognition predominantly utilize traditional machine learning. However, the performance of gait recognition can deteriorate under challenging conditions including environmental occlusion, bulky clothing, and different viewing angles. To provide an effective solution to gait recognition under these conditions, this paper proposes a novel deep learning architecture using Graph Convolutional Neural Network (GCNN) that incorporates residual connections for gait recognition from videos. The optimized feature map of the proposed GCNN architecture exhibits the invariant property to viewing angle and subject's clothing. The residual connection is used to capture both spatial and temporal features of a gait sequence. The kinematic dependency extracted from shallower network layer is propagated to deeper layer using residual connection-based GCNN architecture. The proposed method is validated on CASIA-B gait dataset and outperforms all recent state-of-the-art methods.

show abstract

Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient Rehabilitation

Cited by 13 publications

References 46 publications

Emotion Recognition From Body Movement

Emotion Recognition From Body Movement

Motion Capture Sensor-Based Emotion Recognition Using a Bi-Modular Sequential Neural Network

Residual connection-based graph convolutional neural networks for gait recognition

Contact Info

Product

Resources

About