Flexible Coding of Visual Working Memory Representations during Distraction

Abstract

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM. Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

Simultaneous representation of sensory and mnemonic information in human visual cortex

“…9). This finding is in line with behavioral and neuroimaging demonstrations showing higher fidelity memory recall for similar targets and distractors (Rademaker, Bloem, De Weerd, & Sack, 2015;Lorenc, Sreenivasan, Nee, Vandenbroucke, & D'Esposito, 2018; Supplementary Fig. 2).…”

Different states of priority recruit different neural codes in visual working memory

“…This is because a significant negative reconstruction would require a distributed pattern of activity that differs both from the trained pattern and from baseline, implying an active representation with a code that is different from, in this case, the code with which the AMI was represented during Delay1.2. Furthermore, this finding would implicate early visual cortex in the active representation of the UMI, which is at variance with accounts positing a privileged role for higher-level regions in visual working memory storage during conditions involving shifting attention 16 or distraction 23,24 . Finally, this finding would represent, to our knowledge, the first report of a negative IEM reconstruction as an interpretable index of the state of an active neural representation of stimulus information.…”

Visual working memory representations in visual and parietal cortex do not remap after eye movements

“…Therefore, it could be that the preserved behavioral performance 591 we observed is supported by these IPS representations, rather than the early visual 592 cortex representations. In a more recent study, Lorenc et al (2018) found that the 593 VWM information could be reliably maintained only in early visual cortex without 594 distractors. However, these early visual cortex representations seem to transfer to 595 the IPS representations when distractors were presented, which suggesting a 596 flexible coding of VWM between the early visual cortex and the IPS depending on 597…”