A Cognitive and Unsupervised Map Adaptation Approach to the Recognition of the Focus of Attention from Head Pose

Abstract

Abstract. In this paper, the recognition of the visual focus of attention (VFOA) of meeting participants (as defined by their eye gaze direction) from their head pose is addressed. To this end, the head pose observations are modeled using an Hidden Markov Model (HMM) whose hidden states corresponds to the VFOA. The novelties are threefold. First, contrary to previous studies on the topic, in our set-up, the potential VFOA of a person is not restricted to other participants only, but includes environmental targets (a table and a projection screen), which increases the complexity of the task, with more VFOA targets spread in the pan and tilt (as well) gaze space. Second, the HMM parameters are set by exploiting results from the cognitive science on saccadic eye motion, which allows to predict what the head pose should be given an actual gaze target. Third, an unsupervised parameter adaptation step is proposed which accounts for the specific gazing behaviour of each participant. Using a publicly available corpus of 8 meetings featuring 4 persons, we analyze the above methods by evaluating, through objective performance measures, the recognition of the VFOA from head pose information obtained either using a magnetic sensor device or a vision based tracking system.

Investigating automatic dominance estimation in groups from visual attention and speaking activity

“…As proposed in [11], rather than relying on the common assumption that a person's head is oriented to gaze face-on to a visual target, we rely on findings in psychovision that model the contribution of the head orientation to actual eyein-head gaze shifts, leading to a model of the relationship between a given VFOA target and the head pose. Inference: The VFOA and conversational events are estimated using an iterative process.…”

Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues

“…To set the prior means m k,i , we used our model [28,4], which relies on findings in cognitive sciences about gaze shift dynamics [11,16]. These investigations have shown that, to achieve a gaze rotation towards a visual target from a reference position, only a constant proportion κ of the rotation is made by the head, while the remaining part is made by the eyes.…”

“…Following [28,4], we used κ = (0.5, 0.4). The gaze directions can be computed given the approximate position of the people and objects in the room.…”

Visual activity context for focus of attention estimation in dynamic meetings

“…The cognitive gaze model presented in [4], that relates people's gaze direction to their head pose can be used to predict m k,x j t . The cognitive model assumes that a person k gazes at the direction m where the parameter κ specifies the head pose contribution to the gaze rotation.…”

“…Listeners use their gaze to show their interest, and to request speaking turns [1]. Motivated by gaze shift studies that suggest very strong correlations between the gaze direction, defining people's visual focus of attention (VFOA), and their head pose, research has been conducted to use computer vision techniques to estimate VFOA from head pose in the case where gaze estimation directly from the eyes is impossible [2,3,4]. Yet, most of the studies about VFOA recognition have concentrated on situations where the people are seated during the entire meeting.…”