Ubiquitous Emotion Recognition Using Audio and Video Data

Jannat, Rahatul; Tynes, Iyonna; Lime, Lott La; Adorno, Juan; Canavan, Shaun

doi:10.1145/3267305.3267689

Cited by 29 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An accuracy of 81.39% was obtained for a four expression classification using MMSE DB for training and testing. Another previously mentioned method also utilized MMSE for the AER using audio-visual data obtained from a mobile phone [114].…”

Section: Mmse/bp4d+mentioning

confidence: 99%

A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database

Siddiqui

Dhakal

Yang

et al. 2022

MTI

View full text Add to dashboard Cite

Multimodal human–computer interaction (HCI) systems pledge a more human–human-like interaction between machines and humans. Their prowess in emanating an unambiguous information exchange between the two makes these systems more reliable, efficient, less error prone, and capable of solving complex tasks. Emotion recognition is a realm of HCI that follows multimodality to achieve accurate and natural results. The prodigious use of affective identification in e-learning, marketing, security, health sciences, etc., has increased demand for high-precision emotion recognition systems. Machine learning (ML) is getting its feet wet to ameliorate the process by tweaking the architectures or wielding high-quality databases (DB). This paper presents a survey of such DBs that are being used to develop multimodal emotion recognition (MER) systems. The survey illustrates the DBs that contain multi-channel data, such as facial expressions, speech, physiological signals, body movements, gestures, and lexical features. Few unimodal DBs are also discussed that work in conjunction with other DBs for affect recognition. Further, VIRI, a new DB of visible and infrared (IR) images of subjects expressing five emotions in an uncontrolled, real-world environment, is presented. A rationale for the superiority of the presented corpus over the existing ones is instituted.

show abstract

Section: Mmse/bp4d+mentioning

confidence: 99%

A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database

Siddiqui

Dhakal

Yang

et al. 2022

MTI

View full text Add to dashboard Cite

show abstract

“…Furthermore, a facial expression such as a smile or scowl may communicate something other than emotions [3]. Taking measures of multiple components of emotion may provide a more reliable foundation for inferring emotion [8], as is attempted by several recent ubicomp solutions [4,13,14]. In such work it is important to keep sight of the subjective, experiential component of emotion.…”

Section: Introductionmentioning

confidence: 99%

Making Sense of Emotion-Sensing: Workshop on Quantifying Human Emotions

Tag

Webber

Wadley

et al. 2021

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

View full text Add to dashboard Cite

The global pandemic and the uncertainty if and when life will return to normality have motivated a series of studies on human mental health. This research has elicited evidence for increasing numbers of anxiety, depression, and overall impaired mental well-being. But, the global COVID-19 pandemic has also created new opportunities for research into quantifying human emotions: remotely, contactless, in everyday life. The ubiquitous computing community has long been at the forefront of developing, testing, and building userfacing systems that aim at quantifying human emotion. However, rather than aiming at more accurate sensing algorithms, it is time to critically evaluate whether it is actually possible and in what ways it could be beneficial for technologies to be able to detect user emotions. In this workshop, we bring together experts from the fields of Ubiquitous Computing, Human-Computer Interaction, and Psychology to -long-overdue -merge their expertise and ask the fundamental questions: how do we make sense of emotion-sensing, can and should we quantify human emotions? CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI); Ubiquitous and mobile computing; • Applied computing → Psychology.

show abstract

“…This is also due to the fact that smart emotion recognition technologies are in demand and are introduced around the world, for example, automatic facial expression recognition systems are widely used in medicine [1], psychology [2], education [3], fraud detection [4], driver assistance systems [5], etc. In recent years, more research has focused on the analysis of facial expressions in a video [6][7][8][9] since video can transmit a change in facial expressions over time. Feature extraction is one of the most important steps in facial expression recognition systems by a video stream [10].…”

Section: Introductionmentioning

confidence: 99%

Facial Expression Recognition using Distance Importance Scores Between Facial Landmarks

Ryumina

Karpov

2020

Proceedings of the 30th International Conference on Computer Graphics and Machine Vision (GraphiCon 2020). Part 2

View full text Add to dashboard Cite

In this paper, we present a feature extraction approach for facial expressions recognition based on distance importance scores between the coordinates of facial landmarks. Two audio-visual speech databases (CREMA-D and RAVDESS) were used in the research. We conducted experiments using the Long Short-Term Memory Recurrent Neural Network model in a single corpus and cross-corpus setup with different length sequences. Experiments were carried out using different sets and types of visual features. An accuracy of facial expression recognition was 79.1% and 98.9% for the CREMA-D and RAVDESS databases, respectively. The extracted features provide a better recognition result compared to other methods based on the analysis of facial graphical regions.

show abstract

Ubiquitous Emotion Recognition Using Audio and Video Data

Cited by 29 publications

References 11 publications

A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database

A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI (Visible and InfraRed Image) Database

Making Sense of Emotion-Sensing: Workshop on Quantifying Human Emotions

Facial Expression Recognition using Distance Importance Scores Between Facial Landmarks

Contact Info

Product

Resources

About