Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Abstract

Due to varied personal, social, or even cultural situations, people sometimes conceal or mask their true emotions. These suppressed emotions can be expressed in a very subtle way by brief movements called microexpressions. We investigate human subjects' perception of hidden emotions in virtual faces, inspired by recent psychological experiments. We created animations with virtual faces showing some facial expressions and inserted brief secondary expressions in some sequences, in order to try to convey a subtle second emotion in the character. Our evaluation methodology consists of two sets of experiments, with three different sets of questions. The first experiment verifies that the accuracy and concordance of the participant's responses with synthetic faces matches the empirical results done with photos of real people in the paper by X.-b. Shen, Q. Wu, and X.-l. Fu, 2012, "Effects of the duration of expressions on the recognition of microexpressions," Journal of Zhejiang University Science B, 13(3), 221-230. The second experiment verifies whether participants could perceive and identify primary and secondary emotions in virtual faces. The third experiment tries to evaluate the participant's perception of realism, deceit, and valence of the emotions. Our results show that most of the participants recognized the foreground (macro) emotion and most of the time they perceived the presence of the second (micro) emotion in the animations, although they did not identify it correctly in some samples. This experiment exposes the benefits of conveying microexpressions in computer graphics characters, as they may visually enhance a character's emotional depth through subliminal microexpression cues, and consequently increase the perceived social complexity and believability (Shen, Wu, & Fu, 2012). The second experiment verifies if participants could perceive and identify primary and secondary emotions in virtual faces. The third tries to evaluate the participant's perception of realism, deceit and valence of the emotions. Our results show that most of the participants recognized the foreground (macro) emotion and most of the time they had perceived the presence of the second (micro) emotion in the animations, although they did not identify it correctly in some samples. This experiment exposes the benefits of conveying microexpressions in Computer Graphics characters, as they may visually enhance a character's emotional "depth" through subliminal microexpression cues and consequently increase its perceived social complexity and believability.

Microexpressions in digital humans: perceived affect, sincerity, and trustworthiness

“…Seeing the significant effect of verbal information on affect, the influence seems to be consistent here as well. In general, a happy microexpression was discerned better, aligned with previous research, where participants recognized microexpressions of happiness very well (Queiroz et al, 2014).…”

“…Previous human-computer interaction studies showed that positive facial expressions are generally perceived as more trustworthy (Torre et al, 2020), while extreme facial expressions lead to feelings of eeriness (Mäkäräinen et al, 2014) and can have a negative influence on the perceived sincerity (Stephens et al, 2019). We found that the same holds true for microexpressions, also confirming that happiness is generally better recognized, aligned with Queiroz et al (2014). This helps understand how to create digital humans that users perceive well and, thus, can likely connect with, instead of how they view CAs now: in purely transactional terms (Clark et al, 2019).…”

“…While there is a plethora of studies regarding facial expressions of digital humans with a focus on macroexpressions (Adamo et al, 2019;Loveys et al, 2020;Tinwell et al, 2011;Zell et al, 2019), research is more limited when it comes to exploring how ECA's microexpressions affect user perception. It has been suggested that users can recognize the presence of a microexpression, being able to categorize happiness in particular (Queiroz et al, 2014). Researchers also explored the technical challenges of creating such nonverbal communication, specifically for assisting warfighters in training to distinguish between potential threats and nonthreats (Zielke et al, 2011).…”

Framework and Assessment of Conversational Virtual Humans as Role-players in Simulated Social Encounters with People

“…Limiting this assessment is lack of empirical clarity when the virtual human is part of a learning system containing significant supporting information such as text boxes. The level of contribution of virtual human micro-expressions [63] and verbal tones [64] remain current topics in human-virtual human engagement in role playing research. Virtual human research with the previously mentioned SimSensi and MACH, wearable agents [65,66], virtual peers and companions [67,68] and virtual patients [69,70] may provide further insights.…”

Perception of Charisma, Comfort, Micro and Macro Expressions in Computer Graphics Characters

“…In 2014, Queiroz et al [2] presented a pioneer work where the authors investigate if people perceive the facial micro expressions in a similar way with CG characters than with real people. It is interesting because micro and macro facial expressions can be used to provide more realistic characters in many applications as games and simulations.…”