Facial Expression Synthesis on Robots

Moosaei, Maryam; Hayes, Cory J.; Riek, Laurel D.

doi:10.1145/2701973.2702053

Cited by 4 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interestingly, the human accuracy was roughly 65% on FER-2013, according to Goodfellow et al (2013) , which is about 9% lower than RMN. Due to various factors such as location, perceived gender, and age affecting people’s subjective judgment ( Moosaei and Riek, 2013 ), using an ML-based expression classifier may provide comparable accuracies while being more efficient in terms of time and resources. People should therefore be included in the evaluation stage when the robot is sufficiently developed and can be applied in the context it was created for.…”

Section: Discussionmentioning

confidence: 99%

“…The development has been highly iterative, and thus several prototypes have been built to answer technical questions, as well as being a manifestation of the idea that can be presented to users to gain important feedback ( Auflem et al, 2019 ). However, the challenge of the subjective matter of facial cues and expressions is obtaining actionable and objective data to measure the performance of prototypes by multimodal evaluation ( Moosaei and Riek, 2013 ; Ege et al, 2020 ). This is particularly challenging when evaluating expressive robots due to the resolution and fidelity of the presented prototype being perceived differently, especially when users are unaware of the current state of development.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Facing the FACS—Using AI to Evaluate and Control Facial Action Units in Humanoid Robot Face Development

et al. 2022

View full text Add to dashboard Cite

This paper presents a new approach for evaluating and controlling expressive humanoid robotic faces using open-source computer vision and machine learning methods. Existing research in Human-Robot Interaction lacks flexible and simple tools that are scalable for evaluating and controlling various robotic faces; thus, our goal is to demonstrate the use of readily available AI-based solutions to support the process. We use a newly developed humanoid robot prototype intended for medical training applications as a case example. The approach automatically captures the robot’s facial action units through a webcam during random motion, which are components traditionally used to describe facial muscle movements in humans. Instead of manipulating the actuators individually or training the robot to express specific emotions, we propose using action units as a means for controlling the robotic face, which enables a multitude of ways to generate dynamic motion, expressions, and behavior. The range of action units achieved by the robot is thus analyzed to discover its expressive capabilities and limitations and to develop a control model by correlating action units to actuation parameters. Because the approach is not dependent on specific facial attributes or actuation capabilities, it can be used for different designs and continuously inform the development process. In healthcare training applications, our goal is to establish a prerequisite of expressive capabilities of humanoid robots bounded by industrial and medical design constraints. Furthermore, to mediate human interpretation and thus enable decision-making based on observed cognitive, emotional, and expressive cues, our approach aims to find the minimum viable expressive capabilities of the robot without having to optimize for realism. The results from our case example demonstrate the flexibility and efficiency of the presented AI-based solutions to support the development of humanoid facial robots.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Facing the FACS—Using AI to Evaluate and Control Facial Action Units in Humanoid Robot Face Development

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The human face is a key expressive modality for communicating with others and understanding their intentions and expressions. Facial expressions are a form of visual communication that help to enhance other modalities of communication, such as spoken or gestural language, and enable people to spontaneously communicate important information [60,133]. In clinical settings, healthcare workers use other non-verbal cues to infer patient physiological states, such as pallor, blinking, eye gaze, blushing, and sweating.…”

Section: The Face As a Communication Modality For Robots And Virtual ...mentioning

confidence: 99%

“…The contributions of this work are as follows. First, we extended an FEA system previously developed by our team [133] to improve automatic FACS ratings of facial AUs. The extended FEA system benefits from preprocessing techniques such as noise reduction and facial alignment techniques to diminish the effects of facial deformations, including translation, rotation, and distance to the camera.…”

Section: End-to-end Analysis-modeling-synthesis Framework Developmentmentioning

confidence: 99%

“…Third, we extended an FSA system previously developed by our team [133] for synthesizing realistic, patientlike FEs on both our bespoke RPS head and virtual avatar faces. The method is based on data-driven synthesis, which maps motion from video of an operator/CE onto the face of an embodiment (e.g., virtual avatar or robot).…”

Section: End-to-end Analysis-modeling-synthesis Framework Developmentmentioning

confidence: 99%

See 1 more Smart Citation

Facial Expression Modeling and Synthesis for Patient Simulator Systems: Past, Present, and Future

Pourebadi

Riek

2022

ACM Trans. Comput. Healthcare

Self Cite

View full text Add to dashboard Cite

Clinical educators have used robotic and virtual patient simulator systems (RPS) for dozens of years, to help clinical learners (CL) gain key skills to help avoid future patient harm. These systems can simulate human physiological traits; however, they have static faces and lack the realistic depiction of facial cues, which limits CL engagement and immersion. In this article, we provide a detailed review of existing systems in use, as well as describe the possibilities for new technologies from the human–robot interaction and intelligent virtual agents communities to push forward the state of the art. We also discuss our own work in this area, including new approaches for facial recognition and synthesis on RPS systems, including the ability to realistically display patient facial cues such as pain and stroke. Finally, we discuss future research directions for the field.

show abstract