A Practical Guide for Automated Facial Emotion Classification 1

Stöckli, Sabrina; Schulte‐Mecklenbeck, Michael; Borer, Stefan; Samson, Andrea Christiane

doi:10.4324/9781315160559-14

Cited by 6 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This software uses algorithms to translate the movement of facial features, such as eyes, eye corners, brows, mouth corners, and nose tip, into classifications of emotional valence. Recent work suggests that this automated facial-expression analysis software performs well for detecting emotional states [ 52 , 53 ]. We inspected the aggregated data for the number of occurrences across all respondents, and for any positive, negative, or neutral emotional valence elicited by the visualization.…”

Section: Methodsmentioning

confidence: 99%

A Web Application About Herd Immunity Using Personalized Avatars: Development Study

Hakim¹,

Bettinger²,

Chambers³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

Background Herd immunity or community immunity refers to the reduced risk of infection among susceptible individuals in a population through the presence and proximity of immune individuals. Recent studies suggest that improving the understanding of community immunity may increase intentions to get vaccinated. Objective This study aims to design a web application about community immunity and optimize it based on users’ cognitive and emotional responses. Methods Our multidisciplinary team developed a web application about community immunity to communicate epidemiological evidence in a personalized way. In our application, people build their own community by creating an avatar representing themselves and 8 other avatars representing people around them, for example, their family or coworkers. The application integrates these avatars in a 2-min visualization showing how different parameters (eg, vaccine coverage, and contact within communities) influence community immunity. We predefined communication goals, created prototype visualizations, and tested four iterative versions of our visualization in a university-based human-computer interaction laboratory and community-based settings (a cafeteria, two shopping malls, and a public library). Data included psychophysiological measures (eye tracking, galvanic skin response, facial emotion recognition, and electroencephalogram) to assess participants’ cognitive and affective responses to the visualization and verbal feedback to assess their interpretations of the visualization’s content and messaging. Results Among 110 participants across all four cycles, 68 (61.8%) were women and 38 (34.5%) were men (4/110, 3.6%; not reported), with a mean age of 38 (SD 17) years. More than half (65/110, 59.0%) of participants reported having a university-level education. Iterative changes across the cycles included adding the ability for users to create their own avatars, specific signals about who was represented by the different avatars, using color and movement to indicate protection or lack of protection from infectious disease, and changes to terminology to ensure clarity for people with varying educational backgrounds. Overall, we observed 3 generalizable findings. First, visualization does indeed appear to be a promising medium for conveying what community immunity is and how it works. Second, by involving multiple users in an iterative design process, it is possible to create a short and simple visualization that clearly conveys a complex topic. Finally, evaluating users’ emotional responses during the design process, in addition to their cognitive responses, offers insights that help inform the final design of an intervention. Conclusions Visualization with personalized avatars may help people understand their individual roles in population health. Our app showed promise as a method of communicating the relationship between individual behavior and community health. The next steps will include assessing the effects of the application on risk perception, knowledge, and vaccination intentions in a randomized controlled trial. This study offers a potential road map for designing health communication materials for complex topics such as community immunity.

show abstract

Section: Methodsmentioning

confidence: 99%

A Web Application About Herd Immunity Using Personalized Avatars: Development Study

Hakim¹,

Bettinger²,

Chambers³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…This software uses algorithms to translate the movement of facial features such as eyes, and eye corners, brows, mouth corners, and nose tip into classifications of emotional valence. Recent work suggests that this automated facial-expression analysis software performs well for detecting emotional states [48,49]. We inspected the aggregated data for the number of occurrences across all respondents, and for any positive, negative or neutral emotional valence elicited by the visualization.…”

Section: Discussionmentioning

confidence: 99%

A Web Application About Herd Immunity Using Personalized Avatars: Development Study

Hakim¹,

Bettinger²,

Chambers³

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: ‘Herd immunity’ or ‘community immunity’ refers to the reduced risk of infection among susceptible individuals in a population through the presence and proximity of immune individuals. Recent studies suggest that improving understanding of community immunity may increase intentions to get vaccinated.Objective: We aimed to design a web application about community immunity and optimize it based on viewers’ cognitive and emotional responses.Methods: Our multidisciplinary team developed a web application about community immunity to communicate epidemiological evidence in a personalized way. In our application, people build their own community by creating an avatar representing themselves and 8 other avatars representing people around them; for example, their family or coworkers. The application integrates these avatars in a 2-minute visualization showing how different parameters (e.g., vaccine coverage, contact within communities) influence community immunity. We predefined communication goals, created prototype visualizations, and tested four iterative versions of our visualization in a university-based human-computer interaction laboratory and community-based settings (a cafeteria, two shopping malls, a public library.) Data included psychophysiological measures (eye tracking, galvanic skin response, facial emotion recognition, electroencephalogram) to assess participants’ cognitive and affective (emotional) responses to the visualization, and verbal reports to assess their interpretations of the visualization’s content and messaging.Results and analysis: Participants across all four cycles (N=110 in total) were 62% women and 35% men (4% not reported), with mean age 38 years (SD 17). Fifty-nine percent of participants reported having a university-level education. Iterative changes across the cycles included adding the ability for users to create their own avatars, specific signals about who was represented by the different avatars, using colour and movement to indicate protection or lack of protection from infectious disease, and changes to terminology to ensure clarity for people with varying educational backgrounds. Overall, we observed 3 generalizable findings. First, visualization does indeed appear to be a promising medium for conveying what community immunity is and how it works. Second, by involving multiple users in an iterative design process, it is possible to create a short and simple visualization that clearly conveys a complex topic. Third and finally, evaluating users’ emotional responses during the design process, in addition to their cognitive responses, offers insights and findings that help inform the final design of an intervention. Conclusions: Visualizations with personalized avatars may help people understand their individual roles in population health. Our application showed promise as a method of communicating the relationship between individual behaviour and community health. Next steps will include assessing the effects of the application on risk perception, knowledge, and vaccination intentions in a randomized controlled trial. This study offers a potential roadmap for designing health communication materials for complex topics such as community immunity.

show abstract

“…Instead, it employs a neural network to analyze patterns of wrinkles and crevices that are created by the different action units—an approach that is superior to facial-point-based architectures (iMotions, 2018). In consequence, Emotient FACET provides enhanced accuracy compared to human coders and other algorithms (Stockli et al, 2019). 2…”

Section: Methodsmentioning

confidence: 99%

“…These databases contain validated pictures where trained subjects were photographed displaying facial expressions of certain emotions. Emotient FACET was found to identify facial expressions of emotions from pictures in these datasets with 96% accuracy, which is superior compared to other popular algorithms such as FaceReader (88%) and Affdex (68%; Stockli et al, 2019). Additional validation studies have highlighted the ability of FACET to correctly classify emotions, illustrating strong agreement between FACET and human raters (e.g., Calvo et al, 2018; Krumhuber et al, 2020).…”

mentioning

confidence: 88%

Gender and Counterstereotypical Facial Expressions of Emotion in Crowdfunded Microlending

Davis

Warnick

Anglin

et al. 2021

Entrepreneurship Theory and Practice

View full text Add to dashboard Cite

Crowdfunded microlending research implies that both communal and agentic characteristics are valued. These characteristics, however, are often viewed as being at odds with one another due to their association with gender stereotypes. Drawing upon expectancy violation theory and research on gender stereotypes, we theorize that gender-counterstereotypical facial expressions of emotion provide a means for entrepreneurs to project “missing” agentic or communal characteristics. Leveraging computer-aided facial expression analysis to analyze entrepreneur photographs from 43,210 microloan appeals, we show that women benefit from stereotypically masculine facial expressions of anger and disgust, whereas men benefit from stereotypically feminine facial expressions of sadness and happiness.

show abstract

A Practical Guide for Automated Facial Emotion Classification 1

Cited by 6 publications

References 1 publication

A Web Application About Herd Immunity Using Personalized Avatars: Development Study

A Web Application About Herd Immunity Using Personalized Avatars: Development Study

A Web Application About Herd Immunity Using Personalized Avatars: Development Study

Gender and Counterstereotypical Facial Expressions of Emotion in Crowdfunded Microlending

Contact Info

Product

Resources

About