Spontaneous Facial Expressions and Micro-expressions Coding: From Brain to Face

Dong, Zizhao; Wang, Gang; Lu, Shaoyuan; Li, Jingting; Yan, Wen-Jing; Wang, Sujing

doi:10.3389/fpsyg.2021.784834

Cited by 9 publications

(3 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secondly, the video cameras were not positioned closely enough to capture or encode micro-expressions on the faces of participants. Also, the study of micro-expressions requires coders to have professional knowledge of action unit coding (47), which might be an interesting eld of future research in this context.…”

Section: Future Directionsmentioning

confidence: 99%

Non-verbal elements of rapport during endoscopic procedures - a video ethnographic study

Hoek,

Zweeden,

Velzen

et al. 2024

Preprint

View full text Add to dashboard Cite

Background The environment of the clinical setting of gastrointestinal endoscopy, the cognitive-affective communication of healthcare providers, and the ritual of administering treatments influence patients’ lived experiences. Modern medicine advocates patient-centered care (PCC), and effective therapeutic communication skills are increasingly recognized as essential for creating rapport between healthcare providers and their patients. Rapport can positively influence patient-reported outcomes. However, little is known about how healthcare providers establish rapport in a dynamic setting such as the endoscopy department. The aim of this study was to analyze the nonverbal tools used by healthcare providers working in the endoscopy department to create rapport.Methods We adopted an ethnographic phenomenological approach to study videotaped interactions of endoscopic procedures in a university hospital. Between May 2021 and June 2022, eight procedures were analyzed. The study included healthcare providers (sedation practitioners, endoscopists, and nurses) and patients involved in the endoscopic procedures. The main measures focused on nonverbal communication techniques such as body language as eye contact, gestures, physical touch, and body position.Results Healthcare providers employed nonverbal skills to create rapport, such as sustained eye contact, open postural body positions, physical touch like a soothing touch, and body lean. These elements are commonly seen as key components in one-on-one doctor-patient interactions and the establishment of rapport.Conclusions In conclusion, providing comfort and facilitating the management of emotions are crucial for establishing rapport with patients, whether in one-on-one doctor-patient interactions or within the dynamic endoscopic environment. Findings from this study may contribute to further research and programs aimed at enhancing and implementing therapeutic language and communication skills of healthcare providers in a broader setting.

show abstract

Section: Future Directionsmentioning

confidence: 99%

Non-verbal elements of rapport during endoscopic procedures - a video ethnographic study

Hoek,

Zweeden,

Velzen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Analysis of facial expressions is crucial in psychological analysis ( 7 , 8 ). To calculate facial emotions, psychological theory, mainly based on the circumplex model of emotions was proposed by Russell in 1980 ( 9 ), which considers the division of continuous emotions as a ring in a two-dimensional plane composed of valence and arousal.…”

Section: Introductionmentioning

confidence: 99%

Mental states and personality based on real-time physical activity and facial expression recognition

Huang

Zhai

Song³

et al. 2023

Front. Psychiatry

View full text Add to dashboard Cite

IntroductionTo explore a quick and non-invasive way to measure individual psychological states, this study developed interview-based scales, and multi-modal information was collected from 172 participants.MethodsWe developed the Interview Psychological Symptom Inventory (IPSI) which eventually retained 53 items with nine main factors. All of them performed well in terms of reliability and validity. We used optimized convolutional neural networks and original detection algorithms for the recognition of individual facial expressions and physical activity based on Russell's circumplex model and the five factor model.ResultsWe found that there was a significant correlation between the developed scale and the participants' scores on each factor in the Symptom Checklist-90 (SCL-90) and Big Five Inventory (BFI-2) [r = (−0.257, 0.632), p < 0.01]. Among the multi-modal data, the arousal of facial expressions was significantly correlated with the interval of validity (p < 0.01), valence was significantly correlated with IPSI and SCL-90, and physical activity was significantly correlated with gender, age, and factors of the scales.DiscussionOur research demonstrates that mental health can be monitored and assessed remotely by collecting and analyzing multimodal data from individuals captured by digital tools.

show abstract

“…Manual coding involves trained experts observing and coding facial movements into action units (AUs) based on anatomical features [31]. It is a non-intrusive and highly reliable method that offers considerable face validity.…”

mentioning

confidence: 99%

Exploring Human Emotions: A Virtual Reality-Based Experimental Approach Integrating Physiological and Facial Analysis

Bastida,

Sillaurren,

Loizaga

et al. 2024

MTI

View full text Add to dashboard Cite

This paper researches the classification of human emotions in a virtual reality (VR) context by analysing psychophysiological signals and facial expressions. Key objectives include exploring emotion categorisation models, identifying critical human signals for assessing emotions, and evaluating the accuracy of these signals in VR environments. A systematic literature review was performed through peer-reviewed articles, forming the basis for our methodologies. The integration of various emotion classifiers employs a ‘late fusion’ technique due to varying accuracies among classifiers. Notably, facial expression analysis faces challenges from VR equipment occluding crucial facial regions like the eyes, which significantly impacts emotion recognition accuracy. A weighted averaging system prioritises the psychophysiological classifier over the facial recognition classifiers due to its higher accuracy. Findings suggest that while combined techniques are promising, they struggle with mixed emotional states as well as with fear and trust emotions. The research underscores the potential and limitations of current technologies, recommending enhanced algorithms for effective interpretation of complex emotional expressions in VR. The study provides a groundwork for future advancements, aiming to refine emotion recognition systems through systematic data collection and algorithm optimisation.

show abstract

Spontaneous Facial Expressions and Micro-expressions Coding: From Brain to Face

Cited by 9 publications

References 45 publications

Non-verbal elements of rapport during endoscopic procedures - a video ethnographic study

Non-verbal elements of rapport during endoscopic procedures - a video ethnographic study

Mental states and personality based on real-time physical activity and facial expression recognition

Exploring Human Emotions: A Virtual Reality-Based Experimental Approach Integrating Physiological and Facial Analysis

Contact Info

Product

Resources

About