Analysis of Facial Expressions Explain Affective State and Trust-Based Decisions During Interaction with Autonomy

Dynamics of Trust in Automation and Interactive Decision Making during Driving Simulation Tasks

“…Additional information about participants, task descriptions, and measures can be found in Drnec and Metcalfe (2016) and Gremillion et al (2016). Furthermore, additional analysis of the whole dataset can be found in Neubauer et al (2020) and the previously mentioned articles.…”

“…Trust is only one of multiple factors that lead to a decision to use automation, but it is also difficult to operationalize (Drnec et al, 2016;Neubauer et al, 2020). Some researchers believe that reliance is a more appropriate basis for understanding human-automation dynamics and more easily operationalized (Hoff & Bashir, 2015;Schaefer et al, 2016).…”

“…Reliance is defined as the proportion of time a user allows automation to function without interfering (Lee & See, et al, 2004). Contributions to reliance include general trust in automation, subjective workload, automation quality and type, course features, and personality traits of the operator (Gremillion et al, 2016;Neubauer et al, 2020). All of these factors are important considerations when designing requirements for automation based on the operator's needs.…”

Emotional Design

“…For example, eye tracking data was used to measure attention, GSR was used to measure arousal, and heart rate and heart rate variability were used to measure cognitive workload in the interaction between the driver and automated vehicles (Du et al, 2020). Facial expressions were used to understand participants' dimensional emotional states (Zhou, Kong, et al, 2020) and trust in human-automation interaction (Neubauer et al, 2020). Due to the fact that one measure is not able to reliably measure emotion or cognition, many researchers often use multiple measures together.…”

The Face of Trust: Using Facial Action Units (AUs) as Indicators of Trust in Automation

“…Lee et al found 4 cues, namely touching the face and hands, crossing arms, and leaning back, that together were predictive of lower trust [16]. Neubauer et al showed that face expressivity of basic emotions in tandem with individual factors can be used to model and predict response patterns of users toward automation [17]. Thus, there is much potential in using facial expressions for trust measurement which remains untapped.…”