Wireless Sensing Technology Combined with Facial Expression to Realize Multimodal Emotion Recognition

Dang, Xiaochao; Chen, Zetong; Hao, Zhanjun; Ga, Macidan; Han, Xinyu; Zhang, Xiaolin; Yang, Jie

doi:10.3390/s23010338

Cited by 6 publications

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two papers state directly the use of valence-arousal theory [202], [209]. Regarding machine learning or deep learning choice, we observe both simpler alternatives, with SVM [204], [205], KNN [205]- [207], Random Forest (RF) or DT [205], [206], and deeper models with MLP [206], CNN [202], [206] and CNN + RNN [208], [209]. The size of the dataset of the included papers ranges from 5 subjects [207] to 72 subjects [201], with the subjects involved in a static condition.…”

Section: Radar-based Affective Computingmentioning

confidence: 89%

Machine Learning in RADAR-based Physiological Signals Sensing: A Scoping Review of the Models, Datasets and Metrics

Nocera,

Senigagliesi,

Raimondi

et al. 2024

Preprint

View full text Add to dashboard Cite

In the field of physiological signals monitoring and its applications, non-contact technology is often proposed as a possible alternative to traditional contact devices. The ability to extract information about a patient’s health status in an unobtrusive way, without stressing the subject and without the need of qualified personnel, fuels research in this growing field. Among the various methodologies, RADAR-based non-contact technology is gaining great interest. This scoping review aims to summarize the main research lines concerning RADAR-based physiological sensing and machine learning applications reporting recent trends, issues and gaps with the scientific literature, best methodological practices, employed standards to be followed, challenges, and future directions. After a systematic search and screening, one hundred and ninety two papers were collected following the guidelines of PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses). The included records covered two macro-areas being regression of physiological signals or physiological features (n = 68 papers) and the other a cluster of papers regarding the processing of RADAR-based physiological signals and features applied to four fields of interest, being RADAR-based diagnosis (n = 73), RADAR-based human behaviour monitoring (n = 21), RADAR-based biometrics authentication (n = 18) and RADAR-based affective computing (n = 9). Papers collected under the diagnosis category were further divided, on the basis of their aims: in breath pattern classification (n = 39), infection detection (n = 10), sleep stage classification (n = 9), heart disease detection (n = 8) and quality detection (n = 7). Papers collected under the human behaviour monitoring were further divided based on their aims: fatigue detection (n = 8), human detection (n = 7), human localisation (n = 4), human orientation (n = 2), and activities classification (n = 3).

show abstract

Section: Radar-based Affective Computingmentioning

confidence: 89%

Machine Learning in RADAR-based Physiological Signals Sensing: A Scoping Review of the Models, Datasets and Metrics

Nocera,

Senigagliesi,

Raimondi

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, there has been growing interest in multimodal emotion recognition, driven by advances in deep learning and signal processing techniques. Researchers have proposed and tested various methods [ 23 , 24 , 25 , 26 ] to achieve high accuracy in multimodal emotion recognition, and this field has seen significant progress in terms of both accuracy and real-world applications. Human emotions are subjective and may be influenced by elements such as cultural background, personality, and situational context, making multimodal emotion detection challenging.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Emotion Detection via Attention-Based Fusion of Extracted Facial and Speech Features

Dilnoza

Abdusalomov

Kutlimuratov

et al. 2023

Sensors

View full text Add to dashboard Cite

Methods for detecting emotions that employ many modalities at the same time have been found to be more accurate and resilient than those that rely on a single sense. This is due to the fact that sentiments may be conveyed in a wide range of modalities, each of which offers a different and complementary window into the thoughts and emotions of the speaker. In this way, a more complete picture of a person’s emotional state may emerge through the fusion and analysis of data from several modalities. The research suggests a new attention-based approach to multimodal emotion recognition. This technique integrates facial and speech features that have been extracted by independent encoders in order to pick the aspects that are the most informative. It increases the system’s accuracy by processing speech and facial features of various sizes and focuses on the most useful bits of input. A more comprehensive representation of facial expressions is extracted by the use of both low- and high-level facial features. These modalities are combined using a fusion network to create a multimodal feature vector which is then fed to a classification layer for emotion recognition. The developed system is evaluated on two datasets, IEMOCAP and CMU-MOSEI, and shows superior performance compared to existing models, achieving a weighted accuracy WA of 74.6% and an F1 score of 66.1% on the IEMOCAP dataset and a WA of 80.7% and F1 score of 73.7% on the CMU-MOSEI dataset.

show abstract

Estimation of breathing signal and the respiratory parameters from the speech recordings using honey badger-based modular neural system

Patil,

Naregalkar,

Tondare

et al. 2024

Multimed Tools Appl

View full text Add to dashboard Cite

Wireless Sensing Technology Combined with Facial Expression to Realize Multimodal Emotion Recognition

Cited by 6 publications

References 45 publications

Machine Learning in RADAR-based Physiological Signals Sensing: A Scoping Review of the Models, Datasets and Metrics

Machine Learning in RADAR-based Physiological Signals Sensing: A Scoping Review of the Models, Datasets and Metrics

Multimodal Emotion Detection via Attention-Based Fusion of Extracted Facial and Speech Features

Estimation of breathing signal and the respiratory parameters from the speech recordings using honey badger-based modular neural system

Contact Info

Product

Resources

About