How does gaze to faces support face-to-face interaction? A review and perspective

Abstract

Gaze—where one looks, how long, and when—plays an essential part in human social behavior. While many aspects of social gaze have been reviewed, there is no comprehensive review or theoretical framework that describes how gaze to faces supports face-to-face interaction. In this review, I address the following questions: (1) When does gaze need to be allocated to a particular region of a face in order to provide the relevant information for successful interaction; (2) How do humans look at other people, and faces in particular, regardless of whether gaze needs to be directed at a particular region to acquire the relevant visual information; (3) How does gaze support the regulation of interaction? The work reviewed spans psychophysical research, observational research, and eye-tracking research in both lab-based and interactive contexts. Based on the literature overview, I sketch a framework for future research based on dynamic systems theory. The framework holds that gaze should be investigated in relation to sub-states of the interaction, encompassing sub-states of the interactors, the content of the interaction as well as the interactive context. The relevant sub-states for understanding gaze in interaction vary over different timescales from microgenesis to ontogenesis and phylogenesis. The framework has important implications for vision science, psychopathology, developmental science, and social robotics.

Social-Pragmatic Inferencing, Visual Social Attention and Physiological Reactivity to Complex Social Scenes in Autistic Young Adults

“…Fifth, the stimuli used in the study involved dynamic, complex social situations, yet a passive third-person perspective typical of most structured test situations does not allow for the social participation inherent in real-life interactions. Examination of attention in reallife social interactions may therefore shed light on different aspects of visual social attention, in particular, how gaze is used in interaction (see, e.g., Dindar et al 2017;Gobel et al 2015;Hessels 2020), and may bring out perhaps different information on both competencies and challenges than found in the current study. In the future, such moment-level analyses of visual social attention in real-life interactions would be fruitful in increasing understanding of the role gaze plays in navigating pragmatically complex real-life interactions.…”

Diagnosing Autism Spectrum Disorder Without Expertise: A Pilot Study of 5- to 17-Year-Old Individuals Using Gazefinder

“…As for the preferential paradigm, we have previously confirmed that human figures were more preferentially looked at than non-social figures in typically developing individuals compared to individuals with ASD ( 18 ). To control the spatial preference (e.g., adherence to the right half of the visual field) that may be present in some participants with ASD ( 30 ), two sets of movie clips were duplicated (“Pref E1” vs. “Pref E2,” “Pref F1” vs. “Pref F2”), but the allocation of the targets (human figures) were exchanged horizontally, and inserted as different movie clips. In the other four sets of movies, the appearance of the side (left or right) of the target (human figures) was balanced.…”

Looking behavior and potential human interactions during locomotion

“…Since the pioneering work by Buswell (1935) and Yarbus (1967), it has often been assumed that humans have a bias to look at the bodies and faces of other humans. In a recent review, Hessels (2020) concludes that when no restrictions are placed on observers on where to look or what to do, there is a tendency to fixate humans rather than objects, human faces rather than human bodies, and eyes rather than, for example, the nose or mouth. Looking behavior to human bodies and faces further depends on a number of different factors, including at least (a) the task given to the observer, (b) the content of the portrayed face, and (c) the individual whose looking behavior is investigated.…”

“…1 In this article, we are interested in how the visual system, and in particular looking behavior (where one looks and when), is engaged during such potential interactions. Accurate descriptions of looking behavior across a wide range of human interactions are not only relevant for understanding and modeling human social interaction at large (Hessels, 2020) but also for theories of gaze control and active vision (see, e.g., Land & Furneaux, 1997;Hayhoe, 2000). Moreover, descriptions of how looking behavior supports human interaction have important applications in gaming, human-computer interaction, and social robotics (e.g., Skantze, Hjalmarsson, & Oertel, 2014;Ruhland et al, 2015), as well as in pedestrian and evacuation dynamics (Kitazawa & Fujiyama, 2010) and outdoor lighting design (Fotios, Yang, & Uttley, 2015).…”