WildLab: A naturalistic free viewing experiment reveals previously unknown electroencephalography signatures of face processing

Abstract

Neural mechanisms of face perception are predominantly studied in wellcontrolled experimental settings that involve random stimulus sequences and fixed eye positions. Although powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behaviour, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying non-linear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fixation event-related potentials [fERPs]). Importantly, besides finding a strong correlation between both experiments, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing: This includes category-specific modulation of the event-related potential (ERP)'s amplitude even before fixation onset, as well as adaptation effects across subsequent fixations depending on their history.K E Y W O R D S event-related potentials (ERPs), eye movements, face processing, fixation-related potentials, natural images | INTRODUCTIONThe electroencephalography (EEG) correlates of face processing have been studied widely over the last decades, as faces represent an important stimulus category in our everyday life. Using well-controlled

Active search signatures in a free‐viewing task exploiting concurrent EEG and eye movements recordings

“…The current framework allows to separate the contributions from successive fixations disentangling overlapping components. Recently, some prominent studies have used a similar approach to also separate the contributions of the saccades (Coco et al, 2020; Dimigen & Ehinger, 2021; Gert et al, 2022; Ossandón et al, 2020). We expanded the full model to also include an intercept and the saccade amplitude modelled with a fifth‐order spline associated with the saccade onset.…”

Combining EEG and Eye-Tracking in Virtual Reality - Obtaining Fixation-Onset ERPs and ERSPs

“…Our data confirms this observation, as the generated fERPs show a similar time course to that reported in previous literature (Luck, 2014). As an important note, we did not yet control and correct for temporally overlapping events (Ehinger & Dimigen, 2019; Gert et al, 2022; Henderson et al, 2013) which might affect the overall shape of our fERPs. Specifically, for shorter fixation durations, the following fixation can occur earlier compared to longer fixations, and as a result, the early neuronal components of the following fixation will be superimposed on the late components of the preceding fixation (Dimigen et al, 2011; Henderson et al, 2013).…”

“…In free-viewing experiments, this challenge can be circumvented by defining event or fixation onsets as equivalent to trial onsets (Dimigen et al, 2011), requiring eye-tracking algorithms to capture these accurately and precisely. These allow for the analysis of fixation-onset event-related potentials (fERPs) and event-related spectral perturbations (fERSPs) (Dimigen et al, 2011; Gert et al, 2022), making eye movements classification, including their precise timing, critical. Therefore, as they are better suited to determine the saccade offsets and gaze onsets (Nyström & Holmqvist, 2010), velocity-based algorithms are preferred to be combined with time-sensitive measures, such as EEG.…”

Electrophysiological correlates of face and object perception: A comparative analysis of 2D laboratory and virtual reality conditions

“…Other approaches that focus on conducting their experiments in a natural environment brought the laboratory directly into the real world (e.g., Gert et al., 2022). Although the results obtained in a real‐life environment provide valuable insights into new aspects of face processing, they are also very uncontrollable and make it difficult to reproducibly access physiological correlates.…”