Cortical dynamics of saccade-target selection during free-viewing of natural scenes

Abstract

Acknowledgements:We thank Mia Illman for help in MEG measurements and Veli-Matti Saarinen for help in eye-tracking measurements. The annotations of the stimulus images were created using the Object Labeling Tool from Derek Hoiem, University of Illinois. ABSTRACTNatural visual behaviour entails explorative eye movements, saccades, that bring different parts of a visual scene into the central vision. The neural processes guiding the selection of saccade targets are still largely unknown. Therefore, in this study, we tracked with magnetoencephalography (MEG) cortical dynamics of viewers who were freely exploring novel natural scenes. Overall, the viewers were largely consistent in their gaze behaviour, especially if the scene contained any persons. We took a fresh approach to relate the eyegaze data to the MEG signals by characterizing dynamic cortical representations by means of representational distance matrices. Specifically, we compared the representational distances between the stimuli in the evoked MEG responses with predictions based (1) on the low-level visual similarity of the stimuli (as visually more similar stimuli evoke more similar responses in early visual areas) and (2) on the eye-gaze data. At 50-75 ms after the scene onset, the similarity of the occipital MEG patterns correlated with the low-level visual similarity of the scenes, and already at 75-100 ms the visual features attracting the first saccades predicted the similarity of the right parieto-occipital MEG responses. Thereafter, at 100-125 ms, the landing positions of the upcoming saccades explained MEG responses. These results indicate that MEG signals contain signatures of the rapid processing of natural visual scenes as well as of the initiation of the first saccades, with the processing of the saccade target preceding the processing of the landing position of the upcoming saccade.Humans naturally make eye movements to bring different parts of a visual scene to the fovea where our visual acuity is the best. Tracking of eye gaze can reveal how we make inferences about the content of a scene by looking at different objects, or which visual cues automatically attract our attention and gaze. The brain dynamics governing natural gaze behaviour is still largely unknown. Here we suggest a novel approach to relate eyetracking results with brain activity, as measured with magnetoencephalography (MEG), and demonstrate signatures of natural gaze behaviour in the MEG data already before the eye movements occur.

From Brain–Environment Connections to Temporal Dynamics and Social Interaction: Principles of Human Brain Function

“…Although visual acuity is lower in peripheral than foveal vision, the peripheral visual vision can-by integrating perceptual information over sizeable areas of the visual field-provide powerful summary statistics about various objects and orientations, luminance, and other physical features (Rosenholtz, 2016). For example, summary statistics of shape information from the peripheral retina would explain how a person who fixates the eyes to the center of a screen at the time of a scene-image appearance, tends to direct the first saccade to human faces and other social stimuli located clearly outside the foveal vision (Henriksson et al, 2016). Top-down effects-e.g., from the frontal lobe (Bar et al, 2006)-are expected to help the detection and recognition of relevant objects.…”

Eye Tracking in MEG

“…For tracking eye-movement data for MEG data rejection, the calibration needs to be accurate, so that there will be no misinterpretations about the gaze position, as it effects directly on the data collection (Henriksson et al, 2016;Ramkumar et al, 2016).…”