A Bayesian hierarchy for robust gaze estimation in human–robot interaction

Abstract

In this text, we will present a probabilistic solution for robust gaze estimation in the context of human-robot interaction. Gaze estimation, in the sense of continuously assessing gaze direction of an interlocutor so as to determine his/her focus of visual attention, is important in several important computer vision applications, such as the development of non-intrusive gaze-tracking equipment for psychophysical experiments in neuroscience, specialised telecommunication devices, video surveillance, human-computer interfaces (HCI) and artificial cognitive systems for human-robot interaction (HRI), our application of interest. We have developed a robust solution based on a probabilistic approach that inherently deals with the uncertainty of sensor models, but also and in particular with uncertainty arising from distance, incomplete data and scene dynamics. This solution comprises a hierarchical formulation in the form of a mixture model that loosely follows how geometrical cues provided by facial features are believed to be used by the human perceptual system for gaze estimation. A quantitative analysis of the proposed framework's performance was undertaken through a thorough set of experimental sessions. Results show that the framework performs according to the difficult requirements of HRI applications, namely by exhibiting correctness, robustness and adaptiveness.

Bio-inspired robot perception coupled with robot-modeled human perception

“…While the computer vision community has made significant progress in the area of gaze estimation, the application is typically limited to settings where the head pose is constrained, and the person is close to the camera; examples include phone [16] and tablet [15] use, as well as screen-based settings [23]. However, scenarios that are typically encountered in humanrobot interactions have seen little attention [18,22,20].…”

Improving human–robot interactions in hospitality settings

“…For example, customers could observe service robot head movements (e.g. nod) as a sign of interest or agreement in service interactions (Lanillos et al, 2017).…”

Tracking Gaze and Visual Focus of Attention of People Involved in Social Interaction

“…The correlation between head pose and gaze has also been exploited in [23]. More recently, [24] combined head and eye features to estimate the gaze direction using an RGB-D camera. The method still requires that both eyes are visible.…”