Facial Expression Rendering in Medical Training Simulators: Current Status and Future Directions

Rérolle

et al. 2021

Preprint

Self Cite

Medical training simulators can provide a safe and controlled environment for medical students to practice their physical examination skills. Visual feedback of involuntary pain expressions in response to physical palpation on an affected area of a patient is an important source of information for physicians. However, most existing robotic medical training simulators that can capture physical examination behaviours in real-time cannot display facial expressions or comprise a limited range of patient identities in terms of ethnicity and gender. Together, these limitations restrict the utility of medical training simulators because they do not provide medical students with a representative diversity both of pain facial expressions and face identities, which could result in biased practice. Further, these limitations restrict the utility of such medical simulators to be used to detect and correct early signs of bias in medical training. Here, for the first time, we present a robotic system that can simulate facial expressions of pain in response to palpations, displayed on a range of patient face identities. We use the unique approach of modelling dynamic pain facial expressions using the data-driven psychophysical method of reverse correlation and incorporating the visuo-haptic interactions of users performing palpation to a robot medical simulator. Specifically, participants performed palpation actions on the abdomen phantom of simulated patients, which triggered the real-time display of 6 pain-related facial Action Units (AUs) on a robotic face (MorphFace), each controlled by two pseudo randomly generated transient parameters: rate of change β and activation delay τ. Participants then rated the appropriateness of the facial expression displayed in response to their palpations on a 4-point scale. Each participant (n = 16, 4 Asian female, 4 Asian male, 4 White female and 4 White male) performed 200 palpation trials on 4 patient identities (Black female, Black male, White female and White male) simulated using MorphFace. Results showed that a gradual decrease of β and increase of τ from upper face AUs (around the eyes) to those in the lower face (around the mouth) is rated to be appropriate by all participants. We found that transient parameter values that generated the appropriate pain facial expressions as rated by participants, palpation forces, and delays between palpation actions varied across gender and ethnicity of participant-simulated patient pairs. These findings suggest that gender and ethnicity biases affect the participants’ palpation strategies and their perception of the pain facial expressions displayed on MorphFace. We anticipate our approach could be utilised to generate physical examination models with diverse patient demographic groups to reduce erroneous judgments in medical students, and provide focused training to address these errors.

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Simulating Dynamic Facial Expressions of Pain From Visuo-Haptic Interactions With a Robotic Patient

Rérolle

et al. 2021

Preprint

Self Cite

“…In both SPs and physical mannequins, simulating patients with different demographic identities such as age, gender, and ethnicity is challenging 6 . Some commercially available mannequins such as the Paediatric HAL 7 offer gender and skin colour variations, but options are limited and cannot easily be swapped between training sessions.…”

Section: Introductionmentioning

confidence: 99%

Simulating dynamic facial expressions of pain from visuo-haptic interactions with a robotic patient

Rérolle

et al. 2022

Sci Rep

Self Cite

Medical training simulators can provide a safe and controlled environment for medical students to practice their physical examination skills. An important source of information for physicians is the visual feedback of involuntary pain facial expressions in response to physical palpation on an affected area of a patient. However, most existing robotic medical training simulators that can capture physical examination behaviours in real-time cannot display facial expressions and comprise a limited range of patient identities in terms of ethnicity and gender. Together, these limitations restrict the utility of medical training simulators because they do not provide medical students with a representative sample of pain facial expressions and face identities, which could result in biased practices. Further, these limitations restrict the utility of such medical simulators to detect and correct early signs of bias in medical training. Here, for the first time, we present a robotic system that can simulate facial expressions of pain in response to palpations, displayed on a range of patient face identities. We use the unique approach of modelling dynamic pain facial expressions using a data-driven perception-based psychophysical method combined with the visuo-haptic inputs of users performing palpations on a robot medical simulator. Specifically, participants performed palpation actions on the abdomen phantom of a simulated patient, which triggered the real-time display of six pain-related facial Action Units (AUs) on a robotic face (MorphFace), each controlled by two pseudo randomly generated transient parameters: rate of change $$\beta $$ β and activation delay $$\tau $$ τ . Participants then rated the appropriateness of the facial expression displayed in response to their palpations on a 4-point scale from “strongly disagree” to “strongly agree”. Each participant ($$n=16$$ n = 16 , 4 Asian females, 4 Asian males, 4 White females and 4 White males) performed 200 palpation trials on 4 patient identities (Black female, Black male, White female and White male) simulated using MorphFace. Results showed facial expressions rated as most appropriate by all participants comprise a higher rate of change and shorter delay from upper face AUs (around the eyes) to those in the lower face (around the mouth). In contrast, we found that transient parameter values of most appropriate-rated pain facial expressions, palpation forces, and delays between palpation actions varied across participant-simulated patient pairs according to gender and ethnicity. These findings suggest that gender and ethnicity biases affect palpation strategies and the perception of pain facial expressions displayed on MorphFace. We anticipate that our approach will be used to generate physical examination models with diverse patient demographics to reduce erroneous judgments in medical students, and provide focused training to address these errors.

“…In contrast, virtual human face simulation systems frequently used in computer graphics (CG) such as FACSHuman [17] can render high-fidelity human avatars of different demographics, but cannot respond to physical inputs. The benefits and limitations of using these two approaches were evaluated in greater detail in our recent review in facial expression rendering in medical training simulators [18].…”

Section: Introductionmentioning

confidence: 99%

MorphFace: A Hybrid Morphable Face for a Robopatient

IEEE Robot. Autom. Lett.

et al. 2021

Self Cite

Physicians use pain expressions shown in a patient's face to regulate their palpation methods during physical examination. Training to interpret patients' facial expressions with different genders and ethnicities still remains a challenge, taking novices a long time to learn through experience. This paper presents MorphFace: a controllable 3D physical-virtual hybrid face to represent pain expressions of patients from different ethnicity-gender backgrounds. It is also an intermediate step to expose trainee physicians to the gender and ethnic diversity of patients. We extracted four principal components from the Chicago Face Database to design a four degrees of freedom (DoF) physical face controlled via tendons to span ∼ 85% of facial variations among gender and ethnicity. Details such as skin colour, skin texture, and facial expressions are synthesized by a virtual model and projected onto the 3D physical face via a frontmounted LED projector to obtain a hybrid controllable patient face simulator. A user study revealed that certain differences in ethnicity between the observer and the MorphFace lead to different perceived pain intensity for the same pain level rendered by the MorphFace. This highlights the value of having MorphFace as a controllable hybrid simulator to quantify perceptual differences during physician training.