Recognizing the emotional state of human and virtual instructors

Lawson, Alyssa; Mayer, Richard E.; Adamo-Villani, Nicoletta; Beneš, Bedřich; Lei, Xingyu; Cheng, Justin

doi:10.1016/j.chb.2020.106554

Cited by 58 publications

(47 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Lawson et al (2021a) reported similar findings in a study involving only animated instructors who displayed positive (i.e., happy or content) or negative (i.e., frustrated or bored) emotion. This work is consistent with related work showing that students were able to perceive the emotional stance of an onscreen human or virtual instructor after watching a short video clip from a statistics lesson, particularly whether the instructor was displaying positive (i.e., happy or content) or negative (i.e., frustrated or bored) emotion (Lawson et al, 2021b). Overall, this preliminary work shows that providing both voice cues and embodiment cues can help learners perceive and react to the emotional tone of onscreen agents.…”

Section: Literature Reviewsupporting

confidence: 90%

“…In conjunction, we also test the positivity hypothesis, which posits that people can recognize whether an instructor is displaying positive or negative emotion (Horovitz & Mayer, 2021;Lawson et al, 2021aLawson et al, , 2021b. If people give higher ratings on the happy and content scales for happy and content instructors, and higher ratings on the frustrated and bored ratings for frustrated and bored instructors, this would be evidence for the idea that people are sensitive to the emotional tone of instructors, particularly whether the emotional tone is positive or negative.…”

Section: Objective and Rationalementioning

confidence: 99%

“…Within this field, a more specific subfield concerns affective agents in education, which includes the long-standing study of how features of a pedagogical agent affect how learners perceive the agent's persona, respond to it, and ultimately, how the agent affects their learning outcomes (Baylor et al, 2003;Clarebout et al, 2002;Craig & Schroeder, 2018;Craig et al, 2002;Heidig & Clarebout, 2011;Schroeder et al, 2013). Finally, within the field of affective agents in education, an even more specific subfield concerns how the agent's voice affects the learner's perceptions of the agent and their learning outcome (e.g., Horovitz & Mayer, 2021;Lawson et al, 2021aLawson et al, , 2021bRyu & Ke, 2018). In this study, we focus on the role of the agent's voice and conversational style in online multimedia instructional presentations, as preliminary step towards understanding the role of voice in interactive conversational agents who can engage in a natural language conversation with learner.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Her voice is a recorded human voice, produced by a young adult woman actor completing her studies in Theater. Her gestures, body movement, facial expression, and eye movements were created to mimic a video of the human actor as she delivered the lecture with each of the four emotions (as specified in Lawson et al, 2021aLawson et al, , 2021b.…”

Section: Theory and Predictionsmentioning

confidence: 99%

See 3 more Smart Citations

The Power of Voice to Convey Emotion in Multimedia Instructional Messages

Lawson

Mayer

2021

Int J Artif Intell Educ

Self Cite

View full text Add to dashboard Cite

This study examines an aspect of the role of emotion in multimedia learning, i.e., whether participants can recognize the instructor’s positive or negative emotion based on hearing short clips involving only the instructor’s voice just as well as also seeing an embodied onscreen agent. Participants viewed 16 short video clips from a statistics lecture in which an animated instructor, conveying a happy, content, frustrated, or bored emotion, stands next to a slide as she lectures (agent present) or uses only her voice (agent absent). For each clip, participants rated the instructor on five-point scales for how happy, content, frustrated, and bored the instructor seemed. First, for happy, content, and bored instructors, participants were just as accurate in rating emotional tone based on voice only as with voice plus onscreen agent. This supports the voice hypothesis, which posits that voice is a powerful source of social-emotional information. Second, participants rated happy and content instructors higher on happy and content scales and rated frustrated and bored instructors higher on frustrated and bored scales. This supports the positivity hypothesis, which posits that people are particularly sensitive to the positive or negative tone of multimedia instructional messages.

show abstract

Section: Literature Reviewsupporting

confidence: 90%

Section: Objective and Rationalementioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Theory and Predictionsmentioning

confidence: 99%

See 2 more Smart Citations

The Power of Voice to Convey Emotion in Multimedia Instructional Messages

Lawson

Mayer

2021

Int J Artif Intell Educ

Self Cite

View full text Add to dashboard Cite

show abstract

“…', and the relevant characteristics could then be assembled by adding specific behavioral patterns. This paradigm is still prominently represented across a broad range of research on interface design and robotics [see for example Aeschlimann, Bleiker, Wechner, Gampe, 2020;Cameron et al, 2021;Lawson et al, 2021]. Yet, current theoretical trends in HCI are gradually changing the sociological status quo of the field.…”

Section: Theoretical Shifts In Hcimentioning

confidence: 99%

The Rise of Interactional Multimodality in Human-Computer Interaction

Klowait

Erofeeva

2021

MONITORING OF PUBLIC OPINION

View full text Add to dashboard Cite

The field of human-computer interaction (HCI) investigates the intersection between the design of devices and users. From an early focus on interaction modeling based on psychological experiments, the field has since experienced a shift towards the study of how actual humans interact with autonomous devices. The field became conductive to ethnographic, observational and videographic studies of human-device interaction. Conversation-analytic HCI became possible. That said, this new wave of researchers was never truly able to dethrone the psychological common sense of the field. With recent developments in both the technical-sensorial capabilities and outward actuational range of embodied virtual agents, the field of HCI has once again returned to the question of the sequential unfolding of the interaction between users and intelligent agents, and the multimodal interactional repertoire that is deployed throughout. This review will highlight the situational orientation of high-impact research in the field, and relate it to the cotemporaneous development of ethnomethodological and conversation analytic frameworks. Acknowledgments. The article was prepared in the framework of a research grant funded by the Ministry of Science and Higher Education of the Russian Federation (grant ID: 075-15-2020-908). The article was prepared in cooperation with the Sber (ex. – Sberbank’s) Gamification Lab.

show abstract