Reconstructions of Information in Visual Spatial Working Memory Degrade with Memory Load

Abstract

Summary Working memory (WM) enables the maintenance and manipulation of information relevant to behavioral goals. Variability in WM ability is strongly correlated with IQ [1] and WM function is impaired in many neurological and psychiatric disorders [2, 3], suggesting that this system is a core component of higher cognition. WM storage is thought to be mediated by patterns of activity in neural populations selective for specific properties (e.g., color, orientation, location, motion direction) of memoranda [4–13]. Accordingly, many models propose that differences in the amplitude of these population responses should be related to differences in memory performance [14, 15]. Here, we used functional magnetic resonance imaging (fMRI) and an image reconstruction technique based on a spatial encoding model [16] to visualize and quantify population-level memory representations supported by multivoxel patterns of activation within regions of occipital, parietal and frontal cortex while participants precisely remembered the location(s) of 0, 1, or 2 small stimuli. We successfully reconstructed images containing representations of the remembered – but not forgotten – locations within regions of occipital, parietal, and frontal cortex using delay-period activation patterns. Critically, the amplitude of representations of remembered locations and behavioral performance both decreased with increasing memory load. These results suggest that differences in visual WM performance between memory load conditions are mediated by changes in the fidelity of large-scale population response profiles distributed across multiple areas of human cortex.

Human Dorsolateral Prefrontal Cortex Is Not Necessary for Spatial Working Memory

“…Funahashi et al (1993) used the term "mnemonic scotoma" to draw an analogy between the loss of vision in the contralateral hemifield following damage to early visual cortex and the loss of WM for items in the contralateral hemifield following damage to the monkey dlPFC. Interestingly, hemianopsia (the loss of visual acuity in one hemifield) results in staircase-like sequences of hypometric saccades made to visual targets in the contralesional visual field in humans (Meienberg et al, 1981;Rath-Wilson and Guitton, 2015) similar to those made by the PCS patients to memorized, but not visual, targets. Nonetheless, we must be cautious while concluding that the PCS lesions impaired the ability to store and maintain spatial WM representations because the patients made quick follow-up saccades that corrected their initial error.…”

“…However, spatial WM representations are unlikely to be stored in a single brain area, such as the human dlPFC. Rather, they are stored in networks distributed across multiple topographic maps, cortical and subcortical, including the ones in parietal (Medendorp et al, 2006;Schluppeck et al, 2006;Srimal and Curtis, 2008;Sheremata et al, 2010) and occipital cortex (Saber et al, 2015) that are intact in our patients. We propose that these maps provide the inputs that may compensate for the initial faulty readout from the damaged PCS maps.…”

A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives

“…The receptive fields of voxels in early visual cortex have been characterized as simple two-dimensional Gaussian filters (Dumoulin and Wandell, 2008;Thirion et al, 2006), difference of Gaussians (Zuiderbaan et al, 2012), or as standard multi-parameter Gabor filter banks (Kay et al, 2008). Voxel-wise encoding models have been extended to effects of color (Brouwer and Heeger, 2009), facial identity (Gratton et al, 2013), attention (Sprague and Serences, 2013), working memory (Sprague et al, 2014), numerosity (Harvey et al, 2013), semantics (Huth et al, 2012), and even to other modalities (Schö nwiesner and Zatorre, 2009;Thomas et al, 2015). Encoding models also can incorporate nonlinear processing stages .…”

Visual attention mitigates information loss in small- and large-scale neural codes

“…Each computed stimulus reconstruction contains a representation of the stimulus of interest. Thus, for features like color, orientation, or motion, the results of this procedure reflect a reconstruction of the response across a set of feature-selective populations [9,71,88-90,95-99]; for models based on spatial position, the results reflect reconstructed images of the visual scene viewed or remembered by an observer [75,76,86,100,101]. We call this broad framework whereby patterns of encoding models are inverted in order to reconstruct stimulus representations inverted encoding models (IEM, see glossary).…”