The role of observers' gaze behaviour when watching object manipulation tasks: predicting and evaluating the consequences of action

Abstract

When watching an actor manipulate objects, observers, like the actor, naturally direct their gaze to each object as the hand approaches and typically maintain gaze on the object until the hand departs. Here, we probed the function of observers' eye movements, focusing on two possibilities: (i) that observers' gaze behaviour arises from processes involved in the prediction of the target object of the actor's reaching movement and (ii) that this gaze behaviour supports the evaluation of mechanical events that arise from interactions between the actor's hand and objects. Observers watched an actor reach for and lift one of two presented objects. The observers' task was either to predict the target object or judge its weight. Proactive gaze behaviour, similar to that seen in self-guided action–observation, was seen in the weight judgement task, which requires evaluating mechanical events associated with lifting, but not in the target prediction task. We submit that an important function of gaze behaviour in self-guided action observation is the evaluation of mechanical events associated with interactions between the hand and object. By comparing predicted and actual mechanical events, observers, like actors, can gain knowledge about the world, including information about objects they may subsequently act upon.

Pupils say more than a thousand words: Pupil size reflects how observed actions are interpreted

“…The research described above demonstrates that body language occupies an important role in human social interaction (de Gelder et al, 2010;Van den Stock, Hortensius, Sinke, Goebel, & De Gelder, 2015) and accumulating evidence suggests that naïve observers are able to discriminate subtle body cues to anticipate others' intentions (Ciardo et al, 2017;Koul et al, 2019;Lewkowicz et al, 2015;Sartori et al, 2011;Stapel et al, 2012; see also Cavallo, Koul, Ansuini, Capozzi, & Becchio, 2016 for evidences from a modelling approach). In order to anticipate others' goals, humans integrate information from the others' body movements and eye signals (Flanagan, Rotman, Reichelt, & Johansson, 2013;Rotman, Troje, Johansson, & Flanagan, 2006). The latter plays a fundamental role in this (Baron-Cohen, 1994) as is demonstrated by a study showing that humans attend to another's eyes rather than arms or hands, when predicting others' action goals (Letesson, Grade, & Edwards, 2015).…”

Attentional selection in visual perception, memory and action: a quest for cross-domain integration

“…This provides an important constraint on formal theories of attention, namely the requirement to include affordances, even if seemingly perceptually complex, in the guidance of attention. In this spirit of action-related attention, Flanagan et al [24] test how eye movements depend on the task in action observation. They find that proactive gaze behaviour, similar to the one preparing one's own actions, is elicited if and only if the evaluation of a mechanical event-judging the weight of an object lifted by someone else-is required, when compared with observing the visually identical situation with the task of predicting the choice of an item.…”

“…Nowadays, the efficient and highly precise measurement of temporal and spatial eye-movement parameters [21] has become an increasingly popular way to study attention. In this Theme Issue, eye movements as an overt index of visual selection are prominently featured in various papers [22][23][24][25][26][27][28] and their relation to covert shifts of attention explicitly addressed [29,30].…”

“…First, during the past decades, our knowledge about visual information processing has grown tremendously [31,32]. This enormous progress has been based on experimental paradigms for studying specific functional domains such as visual attention [33], object recognition [34], visual short-term memory [35] or visual-based sensorimotor control rstb.royalsocietypublishing.org Phil Trans R Soc B 368: 20130053 [24,36]. The wealth of research paradigms and empirical data make research within the visual modality an optimal vantage point for further analysis.…”

Visual cues, expectations, and sensorimotor memories in the prediction and perception of object dynamics during manipulation

“…the torque not compensated for at lift-off) and the resulting object tilt. Especially vision might crucially reinforce perception as gaze is predictively directed towards object landmarks during the transition of action phases and aids the judgement of object properties (Flanagan and Johansson 2003;Flanagan et al 2013). As the errors are unpredicted and occur at a crucial phase of the task, they may be attributed as externally caused and thus be highlighted.…”