Eye contact during joint attention with a humanoid robot modulates oscillatory brain activity

Abstract

Eye contact established by a human partner has been shown to affect various cognitive processes of the receiver. However, little is known about humans’ responses to eye contact established by a humanoid robot. Here, we aimed at examining humans’ oscillatory brain response to eye contact with a humanoid robot. Eye contact (or lack thereof) was embedded in a gaze-cueing task and preceded the phase of gaze-related attentional orienting. In addition to examining the effect of eye contact on the recipient, we also tested its impact on gaze-cueing effects (GCEs). Results showed that participants rated eye contact as more engaging and responded with higher desynchronization of alpha-band activity in left fronto-central and central electrode clusters when the robot established eye contact with them, compared to no eye contact condition. However, eye contact did not modulate GCEs. The results are interpreted in terms of the functional roles involved in alpha central rhythms (potentially interpretable also as mu rhythm), including joint attention and engagement in social interaction.

Belief in sharing the same phenomenological experience increases the likelihood of adopting the intentional stance towards a humanoid robot

“…In addition, since the robot would directly address the participants during the Wizardof-Oz interaction, cameras from the robot's eyes were actively recognizing participants' faces, to create mutual gaze between the iCub and participants. Mutual gaze in human-robot interaction has been shown to be a pivotal mechanism that influences human social cognition (Kompatsiari et al, 2018(Kompatsiari et al, , 2021. Facial expressions on the robot were programmed to display the three different emotions (sadness, awe, and happiness) via the YARP emotion interface module.…”

EEG alpha and theta signatures of socially and non-socially cued working memory in virtual reality

“…Here both the social gaze cue and the non-social stick cue engaged the participant prior to cue shift either by looking at them or by pointing at them. It is therefore possible that prior findings that eye contact and joint attention modulated alpha power (Chapman et al, 1975;Kompatsiari et al, 2021;Lachat et al, 2012) are related to the participant attending more when the stimulus is more engaging, either through looking at them, or looking where they look, rather the specific social nature of the stimuli. There was also no difference in the alpha effects between the two cues during the encoding interval.…”

“…2) We predicted alpha and theta power would be more strongly affected by the shift of the social compared to the non-social cue (i.e. Kompatsiari et al, 2021;Lachat et al, 2012). 3) In the social gaze cue condition only, we predicted there would be stronger theta power increases and stronger alpha power decreases for the valid compared to the invalid condition at both the encoding and retrieval intervals.…”

“…Further, higher alpha desynchronization is triggered by a joint attention compared to no joint attention condition in an otherwise identical task (Lachat et al, 2012). Finally, using a humanoid robot in an innovative gaze cuing task, it was found that eye contact prior to gaze shift results in higher alpha desynchronization compared to no eye contact prior to shift (Kompatsiari et al, 2021).…”

“…However, while the use of real people as stimuli is a useful and enlightening method, this can produce limitations in complexity, replicability and the types of behaviours that can be investigated. As an alternative, Wykowska and colleagues have successfully used robots as social stimuli in social cognition research (Kompatsiari et al, 2018(Kompatsiari et al, , 2021Willemse et al, 2018;Wykowska et al, 2016). This demonstrates that it is not always necessary to use real people when studying social phenomenon.…”