Choice anticipation as gated accumulation of sensory expectations

Abstract

Expectations are combined with sensory information when making choices. Some models of the choice process have conceptualized expectations as trial by trial updates to baseline evidence in an accumulator framework. These models have been successful in explaining the influence of choice history across trials on reaction times and choice probabilities, however they do not account for variability in the delay interval within trials. Here, we derive a gated accumulator that models the onset of evidence accumulation as a combination of delayed sensory information and an expectation of sensory timing that changes within trials. To test how the delay intervals interact with trial by trial expectations, we designed a free choice saccade task where participants directed eye movements, as fast as possible, to either of two targets that appeared with variable delays and asynchronies. Despite being instructed to not to anticipate, participants responded prior to target onset on a subset of trials. We reasoned that anticipatory responses may reflect a trade-off between gating the onset of evidence accumulation prior to target onset and releasing this gate as target appearance became more likely. Using a choice history analysis, we found that anticipatory responses were more likely following repeated choices, despite task randomization, suggesting that the balance between anticipatory and sensory responses was driven by an expectation of sensory timing. By fitting the gated accumulator model to reaction time data, we determined that within-trial fluctuations in baseline evidence occurring prior to the onset of the target could explain the joint increase of anticipatory responses and faster sensory-guided responses as the delay period increased. Thus, we conclude that an expectation of sensory timing can shape how the accumulation of choice evidence is gated, allowing us to balance the costs of anticipation with lowering the amount of evidence required to trigger movement.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

“…Although the dynamics of hysteresis may not always be so complex, sequential patterns can emerge from more than just neural activity persisting from previous trials. On the one hand, amplification of hysteresis over time is possible and can be attributed to working memory and its maintenance of past information [211] or instead to accumulating urgency signals [293] and their baseline activation for a response [294]. On the other hand, phenomena such as the diminishing of hysteresis with longer temporal intervals resonate with an account of sustained residual activity [214,216,[295][296][297][298][299][300][301].…”