Modeling human eye movements during immersive visual search

Abstract

The nature of eye movements during visual search has been widely studied in psychology and neuroscience. Virtual reality (VR) paradigms provide an opportunity to test whether computational models of search can predict naturalistic search behavior. However, existing ideal observer models are constrained by strong assumptions about the structure of the world, rendering them impractical for modeling the complexity of environments that can be studied in VR. To address these limitations, we frame naturalistic visual search as a problem of allocating limited cognitive resources, formalized as a meta-level Markov decision process (meta-MDP) over a representation of the environment encoded by a deep neural network. We train reinforcement learning agents to solve the meta-MDP, showing that the agents' optimal policy converges to a classic ideal observer model of search developed for simplified environments. We compare the learned policy with human gaze data from a visual search experiment conducted in VR, finding a qualitative and quantitative correspondence between model predictions and human behavior. Our results suggest that gaze behavior in naturalistic visual search is consistent with rational allocation of limited cognitive resources.

Naturalistic reinforcement learning

“…Virtual reality: Many aspects of naturalistic RL could be best assessed by immersing the participant within a virtual environment. Virtual reality provides a way to do this in a highly effective manner with highly customizable and immersive environments, while also providing for the measurement of complex actions (such as reaching) [ 108 , 109 ]. Portable recordings: New developments in mobile phone tracking and measurement capabilities as well as other portable recording devices offer the opportunity to collect a wide variety of real-time, ecological, and momentary assessments.…”

“…Virtual reality: Many aspects of naturalistic RL could be best assessed by immersing the participant within a virtual environment. Virtual reality provides a way to do this in a highly effective manner with highly customizable and immersive environments, while also providing for the measurement of complex actions (such as reaching) [ 108 , 109 ].…”

Motor “laziness” constrains fixation selection in real-world tasks

“…Some work already exists in trying to approximate user effort with oculomotor energy computed by eye plant models [46]. In general, models of gaze behavior at many levels of abstraction can incorporate oculomotor costs by considering fixation eccentricity as a proxy for effort (for head-fixed observers) and treating this as a (negative) reward, including active inference models of gaze behavior [3] and reinforcement learning models [4]. Deep learning models may have already learned this cost implicitly, hence why some machine learning researchers (correctly) try to factor out the predictiveness of the center bias on its own when evaluating saliency/gaze prediction deep learning models [47].…”

“…Fixation selection can be described as an active inference process which maximizes reward while minimizing costs for the agent. In this framework, gaze shifts and maintenance of fixation both serve to reduce uncertainty about task-relevant properties of the environment, preventing sub-optimal actions [1][2][3][4]. As an example, fixating an ambiguous road sign while driving enables reading and heeding the sign's text, "Detour", reducing the total time taken to reach a destination (a type of reward).…”

“…Decades of research on fixation selection and gaze behavior in psychology and neuroscience has generated many important insights, including the discovery of reward sensitivity of neural oculomotor circuitry, as well as the conceptual advances of saliency maps, inhibition-of-return, reinforcement learning, and active inference in understanding and modelling gaze behavior [2–4, 11, 12]. Perhaps the most general properties of fixation behavior are (1) the center bias, the tendency for the eye to remain mostly near the head orientation / center of the orbit, and return phenomena like (2) spatial inhibition-of-return, the tendency for observers to more often gaze at new locations than recently visited ones, and (3) temporal inhibition-of-return, longer fixation durations preceding a return than gazing at a new location.…”