A Critique of Automated Approaches to Code Facial Expressions: What Do Researchers Need to Know?

Cross, Marie P.; Acevedo, Amanda M.; Hunter, Judy

doi:10.1007/s42761-023-00195-0

Cited by 6 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fourth, on a methodological level, using facial expressions as indicators of emotional experiences, and specifically, applying automated facial expression analysis algorithms is not without criticism (Barrett et al, 2019;Cross et al, 2023). Concerning the debate around the congruence of facial expressions with the underlying emotional experiences, we do not propose that subjective emotional states can be inferred directly from an individual's facial expressions.…”

Section: Limitationsmentioning

confidence: 95%

Marx, Sachs, Frenzel, & Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Marx,

Sachs

2024

Preprint

View full text Add to dashboard Cite

Over the last years, many aspects of people‘s lives have been moved to digitally supported environments. So far, the impact of these changes on their emotions and interpersonal processes remains largely unclear. As one essential and prevalent emotional process in social interaction, emotional contagion comprises the transmission of emotions between two or more individuals. In previous research, it has been studied in various interactive face-to-face contexts (e.g., teachers/students or psychotherapists/patients). In this study, we aimed to add to these findings by examining emotional contagion in dyadic online video conferences. We applied a lab-based experimental approach involving two participants interacting with each other via synchronized computers. In a structured interaction paradigm, they (N = 104 in n = 52 dyads) were prompted to talk to each other about recent personal experiences that made them angry, happy, and sad (in 3 conditions) while being filmed by webcams. The participants’ facial expressions during the interaction were analyzed using automated facial expression analysis software, and they retrospectively reported their subjective emotional experiences after each condition. Repeated measures analyses provided evidence of emotional contagion during the video conferences, as listeners reported to experience the speakers’ target emotions (anger, joy, sadness) significantly more frequently than any other emotion. We additionally employed cross-recurrence quantification analysis and found evidence for above chance co-occurrence of facially expressed anger and joy in both interaction partners. These findings point towards the existence of facial mimicry in online video conferences – a process that has been proposed as underlying mechanism of emotional contagion.

show abstract

Section: Limitationsmentioning

confidence: 95%

Marx, Sachs, Frenzel, & Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Marx,

Sachs

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Such responses are more aligned with the human perception of facial expressions that have been conceptualized as a manifold representation of varying degrees of emotions (e.g., a face may be perceived as 51% anger and 49% disgust [ 101 ]). In comparison, most existing emotion models rely on majority voting to determine the single most representative label [ 102 ]. Although such approaches may enhance the overall reliability of emotion prediction, they may exclude other relevant but less dominant emotions and fail to detect compound emotions [ 103 ].…”

Section: Advancing Naturalistic Affective Science With Multimodal Lar...mentioning

confidence: 99%

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

Bian,

Küster,

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.

show abstract

The Future of Affective Science: Introduction to the Special Issue

Shiota,

Camras,

Adolphs

2023

Affec Sci

View full text Add to dashboard Cite

A Critique of Automated Approaches to Code Facial Expressions: What Do Researchers Need to Know?

Cited by 6 publications

References 32 publications

Marx, Sachs, Frenzel, & Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Marx, Sachs, Frenzel, & Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

The Future of Affective Science: Introduction to the Special Issue

Contact Info

Product

Resources

About

A Critique of Automated Approaches to Code Facial Expressions: What Do Researchers Need to Know?

Cited by 6 publications

References 32 publications

Marx, Sachs, Frenzel, &amp; Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Marx, Sachs, Frenzel, &amp; Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

The Future of Affective Science: Introduction to the Special Issue

Contact Info

Product

Resources

About

Marx, Sachs, Frenzel, & Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data

Marx, Sachs, Frenzel, & Schweizer [Preprint] Emotional contagion in dyadic online video conferences - Empirical evidence based on self-report and facial expression data