Recognizing the Visual Focus of Attention for Human Robot Interaction

Abstract

Abstract. We address the recognition of people's visual focus of attention (VFOA), the discrete version of gaze that indicates who is looking at whom or what. As a good indicator of addressee-hood (who speaks to whom, and in particular is a person speaking to the robot) and of people's interest, VFOA is an important cue for supporting dialog modelling in Human-Robot interactions involving multiple persons. In absence of high definition images, we rely on people's head pose to recognize the VFOA. Rather than assuming a fixed mapping between head pose directions and gaze target directions, we investigate models that perform a dynamic (temporal) mapping implicitly accounting for varying body/shoulder orientations of a person over time, as well as unsupervised adaptation. Evaluated on a public dataset and on data recorded with the humanoid robot Nao, the method exhibit better adaptivity and versatility producing equal or better performance than a state-of-the-art approach, while the proposed unsupervised adaptation does not improve results.

Intelligent Presentation Skills Trainer Analyses Body Movement

“…Instead, it more concerns whether the audience can perceive that they are being addressed. In many applications, instead of localizing gaze exactly, head/face orientation was used as the effective approximation for subjects focus target [18,19]. The experiment in [20] also proved that, head orientation was the reliable indication of the visual focus of attention in 89% of the time.…”

Human Behavior Understanding for Robotics

“…One of them is the understanding the visual focus of attention of humans while interacting with robots. This is addressed in this volume [64].…”

“…Especially, forming joint attention through modeling the gaze of a human can be very useful in human-robot collaboration scenarios or when a human teacher teaches tasks or concepts involving the objects in the environment [70,64]. In [70], object saliency is used in conjunction with head pose estimates to allow a humanoid robot to determine the visual focus of attention of the interacting human, while in [64] a fixed mapping between head pose directions and gaze target directions was not assumed, and models are investigated that perform a dynamic (temporal) mapping implicitly accounting for varying body/shoulder orientations of a person over time, as well as unsupervised adaptation.…”

Detecting conversational gaze aversion using unsupervised learning

“…In [14] the SVM-based approach is improved upon with the use of Latent-Dynamic Conditional Random Fields (LDCRFs). Methods of deducing visual focus of attention (VFOA) [15], [16], [17] could also be used to infer gaze aversion. Many VFOA methods rely on head orientation estimation to distinguish the focus of attention in multi-party meeting scenarios.…”