Optimal and human eye movements to clustered low value cues to increase decision rewards during search

Abstract

Rewards have important influences on the motor planning of primates and the firing of neurons coding visual information and action. When eye movements to a target are differentially rewarded across locations, primates execute saccades towards the possible target location with the highest expected value, a product of sensory evidence and potentially earned reward (saccade to maximum expected value model, sMEV). Yet, in the natural world eye movements are not directly rewarded. Their role is to gather information to support subsequent rewarded search decisions and actions. Less is known about the effects of decision rewards on saccades. We show that when varying the decision rewards across cued locations following visual search, humans can plan their eye movements to increase decision rewards. Critically, we report a scenario for which five of seven tested humans do not preferentially deploy saccades to the possible target location with the highest reward, a strategy which is optimal when rewarding eye movements. Instead, these humans make saccades towards lower value but clustered locations when this strategy optimizes decision rewards consistent with the preferences of an ideal Bayesian reward searcher that takes into account the visibility of the target across eccentricities. The ideal reward searcher can be approximated with a sMEV model with pooling of rewards from spatially clustered locations. We also find observers with systematic departures from the optimal strategy and inter-observer variability of eye movement plans. These deviations often reflect multiplicity of fixation strategies that lead to near optimal decision rewards but, for some observers, it relates to suboptimal choices in eye movement planning.

Control of gaze while walking: Task structure, reward, and uncertainty

“…In the present conceptualization, gaze target is chosen to reduce uncertainty in order to make better decisions about walking direction, and in the Reinforcement Learning framework, walking direction is chosen to maximize future expected discounted reward that has been learned through experience. As pointed out by Eckstein, Schoonveld, Zhang, Mack, & Akbas (2015), and others (Gottlieb et al, 2012; Gottlieb, Hayhoe, Hikosaka, & Rangel, 2014) eye movements themselves are not directly rewarded in the context of behavior, unlike many of the neurophysiological experiments as well as some of the psychophysical experiments (e.g., Navalpakkam et al, 2010, Schutz et al, 2012; Stritzke et al, 2009). Eckstein et al (2015) argue that, at least in some cases, it is the decision informed by the eye movement that gets the reward, not the eye movement per se.…”

Parietal neurons encode expected gains in instrumental information

“…Although replicating the essential features of instrumental sampling requires relatively complex sequential paradigms (10,29), extensive analyses and control conditions ruled out confounds that may arise in such paradigms. We discuss the implication of the findings from the perspectives of the valuebased and priority-based interpretations of LIP function.…”

“…A second key question is whether EIG is computed dynamically based on the uncertainty of each forthcoming action or relies on long-term estimates of average validity. A dynamic "look ahead" mechanism affords high flexibility, but it may be computationally expensive and may not be used consistently in all behavioral contexts (10,29). An important question for future research is to what extent EIG computations are flexible and responsive to rapid changes in context, or rely on stored validity representations to produce routine-based sampling policies (13,41,42).…”

Domain Specificity of Oculomotor Learning after Changes in Sensory Processing

“…Face processing has its own dedicated system in the brain (Kanwisher et al, 1997). Face recognition is a highly practiced task for which humans develop fixation strategies that remain very consistent across time (Peterson and Eckstein, 2013b;Mehoudar et al, 2014) and show a consistent optimality and rigidity Eckstein, 2013b, 2014) that is rarely present in simpler artificial laboratory tasks (Verghese, 2012;Ackermann and Landy, 2013;Eckstein et al, 2015).…”

“…Face recognition is a highly practiced task for which humans develop fixation strategies that remain very consistent across time (Peterson and Eckstein, 2013b;Mehoudar et al, 2014) and show a consistent optimality and rigidity Eckstein, 2013b, 2014) that is rarely present in simpler artificial laboratory tasks (Verghese, 2012;Ackermann and Landy, 2013;Eckstein et al, 2015). The frequently used fixation strategy with faces might give rise to special fixation-specific representations in face-related brain areas.…”