Use of affect context in dyadic interactions for continuous emotion recognition

Fatima, Syeda Narjis; Erzin, Engin

doi:10.1016/j.specom.2021.05.010

Cited by 4 publications

(2 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this regard, there are different methods in the context of human emotions recognition. For instance, by tracking implicit parameters, including, speech recognition [13], [14] facial expression recognition [15]- [17], physiological means [18], [19] body gesture recognition [20] or multimodal or fusion means [12], [21], [22]. While some sensory cues such as speech, body language, and physiological measures, may not yet be realistically and decipherable by a computer as effortless as the human does, the facial expression could be applicable.…”

Section: Introductionmentioning

confidence: 99%

Recognizing facial emotions for educational learning settings

Kingsley

Inyang

Msugh

et al. 2022

IJRA

View full text Add to dashboard Cite

Educational learning settings exploit cognitive factors as ultimate feedback to enhance personalization in teaching and learning. But besides cognition, the emotions of the learner which reflect the affective learning dimension also play an important role in the learning process. The emotions can be recognized by tracking explicit behaviors of the learner like facial or vocal expressions. Despite reasonable efforts to recognize emotions, the research community is currently constraints by two issues, namely : i) the lack of efficient feature descriptors to accurately represent and prospectively recogniz e (detecting) the emotions of the learner ; ii) lack of contextual datasets to benchmark performances of emotion recognizers in the learning - speci fic scenarios, resulting in poor generalizations. This paper presents a facial emotion recognition technique (FERT). The FERT is realized through results of preliminary analysis across various facial feature descriptors. Emotions are classified using the m ultiple kernel learning (MKL) method which reportedly possesses good merits. A contextually relevant simulated learning emotion ( SLE ) dataset is introduced to validate the FERT scheme. Recognition performance of the FERT scheme generalizes to 90.3% on the SLE dataset. On more popular but noncontextually datasets, the scheme achi e ved 90.0% and 82.8% respectively extended Cohn Kanade (CK+) and acted facial expressions in the wild ( AFEW ) datasets. A test for the null hypothesis that there is no significant difference in the performances accuracies of the descriptors rather proved otherwise ( x2 = 14 . 619 , df = 5 , p = 0 . 01212 ) for a model considered at a 95% confidence interval.

show abstract

Section: Introductionmentioning

confidence: 99%

Recognizing facial emotions for educational learning settings

Kingsley

Inyang

Msugh

et al. 2022

IJRA

View full text Add to dashboard Cite

show abstract

“…However, the interaction between Individual Emotion and Conversation Valence was not found in the hypothesised form. The current study found that for Anger, Disgust and Surprise, the models just It is also possible that the conversation topic and the emotional states of the participants in each dyad influenced the emotional states of each other (Fatima & Erzin, 2021). Each participant's data were stacked on each other in the machine learning models.…”

Section: The Interaction Between Context and Individual Emotionsmentioning

confidence: 96%

Theory Based Modelling of the Relationship Between Emotional States and Bodily Activity Using Machine Learning

Goldsack

View full text Add to dashboard Cite

The current study uses machine learning with a theoretically derived approach to investigate the relationship between emotion and bodily activity. Previous literature focuses on the accuracy of machine learning models when predicting emotional states. The studies lack theoretical grounding in emotion literature justifying the appropriateness of the chosen machine learning model architectures in predicting emotional states. The current studies’ machine learning models use data from dyadic interactions in positively and negatively valenced conversations. The series of machine learning models apply the same architecture to predict a person’s emotional state using bodily activity. To achieve this the models are trained on multiple measures of a person’s emotional state and bodily activity across differently valenced conversations. For example, a model is trained/tested on emotion measure #1, bodily activity measure #1 and positive conversations. In contrast, another model is trained and tested on emotion measure #1, bodily activity measure #2, and negative conversations. The Test Error resulting from the machine learning models was then statistically tested to answer various questions. The machine learning models trained/tested on emotion measures including the arousal and valence of an emotional state performed the better than those including an emotion measure of how arousal changes over time. The models found that a person’s physiological activity most accurately predicts their emotional state, compared to predicting with the person’s movements. The models trained/tested on negatively valenced conversations predicted people’s emotional states better than those on positively valenced conversations. The results show that not all emotion measures are equal, and the valence and arousal of an emotional state most accurately represent the emotional state. The results replicated the close relationship between emotion and physiological activity. Finally, the context valence alters the accuracy of predicting a person’s emotional state – knowing someone’s situation provides information about their emotional state.

show abstract

A Computation Model to Estimate Interaction Intensity through Non-verbal Behavioral Cues: A Case Study of Intimate Couples under the Impact of Acute Alcohol Consumption

Yu,

Crane,

Chen

et al. 2024

ACM Trans. Comput. Healthcare

View full text Add to dashboard Cite

This work introduced a novel analysis method to estimate interaction intensity, i.e., the level of positivity/negativity of an interaction, for intimate couples (married and heterosexual) under the impact of alcohol, which has great influences on behavioral health. Non-verbal behaviors are critical in interpersonal interactions. However, whether computer vision-detected non-verbal behaviors can effectively estimate interaction intensity of intimate couples is still unexplored. In this work, we proposed novel measurements and investigated their feasibility to estimate interaction intensities through machine learning regression models. Analyses were conducted based on a conflict-resolution conversation video dataset of intimate couples before and after acute alcohol consumption. Results showed the estimation error was at the lowest in the no-alcohol state but significantly increased if the model trained using no-alcohol data was applied to after-alcohol data, indicating that alcohol altered the interaction data in the feature space. While training a model using rich after-alcohol data is ideal to address the performance decrease, data collection in such a risky state is challenging in real life. Thus, we proposed a new State-Induced Domain Adaptation (SIDA) framework, which allows for improving estimation performance using only a small after-alcohol training dataset, pointing to a future direction of addressing data scarcity issues.

show abstract

Use of affect context in dyadic interactions for continuous emotion recognition

Cited by 4 publications

References 39 publications

Recognizing facial emotions for educational learning settings

Recognizing facial emotions for educational learning settings

Theory Based Modelling of the Relationship Between Emotional States and Bodily Activity Using Machine Learning

A Computation Model to Estimate Interaction Intensity through Non-verbal Behavioral Cues: A Case Study of Intimate Couples under the Impact of Acute Alcohol Consumption

Contact Info

Product

Resources

About