Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Abstract

Behavioural reports of sensory information are biased by stimulus history. The nature and direction of such serial-dependence biases can differ between experimental settings – both attractive and repulsive biases towards previous stimuli have been observed. How and when these biases arise in the human brain remains largely unexplored. They could occur either via a change in sensory processing itself, post-perceptual maintenance or decision-making processes, or both. Here, we analysed behavioural and magnetoencephalographic data from a working-memory task in which participants were sequentially presented with two randomly oriented gratings, one of which was cued for recall at the end of the trial. Behavioural responses showed evidence for two distinct biases: 1) a within-trial repulsive bias away from the previously encoded orientation on the same trial, and 2) a between-trial attractive bias towards the task-relevant orientation on the previous trial. Multivariate classification of stimulus orientation revealed that neural representations during stimulus encoding were biased away from the previous grating orientation, regardless of whether we considered the within- or between-trial prior orientation – despite opposite effects on behaviour. These results suggest that repulsive biases occur at the level of sensory processing and can be overturned at post-perceptual stages to result in attractive biases in behaviour.

Attractive serial dependence overcomes repulsive neuronal adaptation

“…This is consistent with our model, and it suggests that some representation of information about stimulus history should be a minimum requirement for an aware decoding scheme. The identity of the previous stimulus for spatial position and angle has been shown to be decodable from the spiking activity of single units in the FEF and posterior parietal cortex (PPC) as well as large-scale activity patterns in human EEG and MEG [31][32][33]54,58,65]. We additionally demonstrate that information about the previous trial is encoded in patterns of fMRI activity in human visual cortex (Fig 2F ), but not in a sensory-like code (S4A and S4B Fig) . These signals could potentially be represented concurrently with representations of the current stimulus in the same populations of sensory neurons but in orthogonal codes analogous to what has been found for sequentially encoded items in primate prefrontal cortex and human EEG [66,67].…”

“…However, our observation of repulsive biases starting in V1 and persisting across later visual areas suggests that bottom-up adaptation may be a viable alternative explanation (which the authors also acknowledged). Further support for this account comes from a recent magnetoencephalography (MEG) study showing that representations were repelled from past stimuli both within the current trial and from the previous trial [ 58 ]. As in our study, this neural repulsion contrasts with attractive behavioral biases to the previous stimulus, suggesting that sensory representations do not directly shape behavior even in simple sensory paradigms [ 50 ].…”

“…This is consistent with our model, and it suggests that some representation of information about stimulus history should be a minimum requirement for an aware decoding scheme. The identity of the previous stimulus for spatial position and angle has been shown to be decodable from the spiking activity of single units in the FEF and posterior parietal cortex (PPC) as well as large-scale activity patterns in human EEG and MEG [ 31 – 33 , 54 , 58 , 65 ]. We additionally demonstrate that information about the previous trial is encoded in patterns of fMRI activity in human visual cortex ( Fig 2F ), but not in a sensory-like code ( S4A and S4B Fig ).…”

Sensory readout accounts for adaptation

“…However, our observation of repulsive biases starting in V1 and persisting across later visual areas suggests that bottom-up adaptation may be a viable alternative explanation (which the authors also acknowledged). Further support for this account comes from a recent MEG study showing that representations were repelled from past stimuli both within the current trial and from the previous trial (Hajonides et al, 2021). As in our study, this neural repulsion contrasts with attractive behavioral biases to the previous stimulus, suggesting sensory representations do not directly shape behavior even in simple sensory paradigms (Siegle et al, 2021).…”

“…This is consistent with our model, and it suggests that some representation of information about stimulus history should be a minimum requirement for an aware decoding scheme. The identity of the previous stimulus for spatial position and angle has been shown to be decodable from the spiking activity of single units in the frontal eye field (FEF) and large-scale activity patterns in human EEG and MEG (Papadimitriou et al, 2016; Fornaciai and Park, 2018; Bae and Luck, 2019; Bae, 2021; Hajonides et al, 2021). We additionally demonstrate that information about the previous trial is encoded in patterns of fMRI activity in human visual cortex (Figure 2F), but not in a sensory-like code (Figure S4AB).…”