Using visual information to guide behaviour requires storage in a temporary buffer, known as visual short-term memory (VSTM), that sustains attended information across saccades and other visual interruptions. There is growing debate on whether VSTM capacity is limited to a fixed number of objects or whether it is variable. Here we report four experiments using functional magnetic resonance imaging that resolve this controversy by dissociating the representation capacities of the parietal and occipital cortices. Whereas representations in the inferior intra-parietal sulcus (IPS) are fixed to about four objects at different spatial locations regardless of object complexity, those in the superior IPS and the lateral occipital complex are variable, tracking the number of objects held in VSTM, and representing fewer than four objects as their complexity increases. These neural response patterns were observed during both VSTM encoding and maintenance. Thus, multiple systems act together to support VSTM: whereas the inferior IPS maintains spatial attention over a fixed number of objects at different spatial locations, the superior IPS and the lateral occipital complex encode and maintain a variable subset of the attended objects, depending on their complexity. VSTM capacity is therefore determined both by a fixed number of objects and by object complexity.

Recent studies have provided conflicting accounts regarding where in the human brain visual short-term memory (VSTM) content is stored, with strong univariate fMRI responses reported in superior intraparietal sulcus (IPS) but robust multivariate decoding reported in occipital cortex. Given the continuous influx of information in everyday vision, VSTM storage under distraction is often required. We found that neither distractor presence nor predictability during the memory delay affected behavioral performance. Similarly, superior IPS exhibited consistent decoding of VSTM content across all distractor manipulations and had multivariate responses that closely tracked behavioral VSTM performance. However, occipital decoding of VSTM content was significantly modulated by distractor presence and predictability. Furthermore, we found no effect of target-distractor similarity on VSTM behavioral performance, further challenging the role of sensory regions in VSTM storage. Overall, consistent with previous univariate findings, these results show that superior IPS, not occipital cortex, plays a central role in VSTM storage.

To explain how multiple visual objects are attended and perceived, we propose that our visual system first selects a fixed number of about four objects from a crowded scene based on their spatial information (object individuation) and then encode their details (object identification). We describe the involvement of the inferior intra-parietal sulcus (IPS) in object individuation and the superior IPS and higher visual areas in object identification. Our neural object-file theory synthesizes and extends existing ideas in visual cognition and is supported by behavioral and neuroimaging results. It provides a better understanding of the role of the different parietal areas in encoding visual objects and can explain various forms of capacity-limited processing in visual cognition such as working memory.

No abstract

The present study investigated object-based feature encoding in visual short-term memory for 2 features within the same dimension that occur on different parts of an object. Using the change-detection paradigm, this experiment studied objects with 2 colors and objects with 2 orientations. Participants found it easier to monitor 1 rather than both features of such objects, even when decision noise was properly controlled for. However, no object-based benefit was observed for encoding the 2 features of each object that were of the same dimension. When similar stimuli were used but the 2 features of each object were from different dimensions (color and orientation), an object-based benefit was observed. These results thus impose a major constraint on object-based feature encoding theories by showing that only features from different dimensions can benefit from object-based encoding.How are features of objects perceived and retained? In an early study by Allport (1971), colored numerals inside colored shapes were presented briefly for recall of one or more features. It was found that report of a form feature (e.g., the shapes) was not affected by whether a color feature (colors of the shapes or numerals) was also reported, but was negatively affected by the report of another form feature (e.g., the numerals). A second study by Wing and Allport (1972) confirmed these findings and found that the report of both spatial frequency and orientation did not affect either report, but the report of two orientation features (grating orientation and an orientation of a transverse break in the lines of the grating) interfered with each report significantly. These authors argued that perceptual analysis occurs in systems of analyzers, each dealing with a specific feature dimension such as color or orientation. As such, two features from different dimensions can be encoded in parallel without mutual interference, but two features from the same dimension will have to share the same analyzer and cannot be encoded without interference (see also Treisman, 1969). Duncan (1984) questioned the above conclusion and argued that feature grouping by objects plays a more important role in perception. He asked participants to report two of four features from four independent dimensions (size, tilt, texture, and the location of a gap), either located on the same object or on different objects. He found that two features located on the same object could be reported more readily than two features located on different objects. In a later study in which two letters were presented, Duncan (1993) asked participants to report (a) the size and shape of one letter, (b) the same attribute (size or shape) of two letters, (c) the shape of one letter and size of the other, or (d) both attributes of both letters. He found that report of (a) was much better than that of (b), (c), or (d) and that the differences among (b), (c), and (d) were not significant. In a second experiment, Duncan (1993) presented two objects, each containing orientation, length, and fr...

Can we find an object-based encoding benefit in visual short-term memory (VSTM) when the features to be remembered are from different parts of an object? Using object parts defined by either figure-ground separation or negative minima of curvature, results from five experiments in which the visual change detection paradigm was used showed that the object-based encoding benefit in VSTM is modulated by how features are assigned to parts of an object: Features are best retained when the color and shape features to be remembered belong to the same part of an object. Although less well retained in comparison, features from different parts of an object are still better remembered than features from spatially separated objects. An object-based feature binding therefore exists even when the color and shape features to be remembered are from different parts of an object.

It has previously been reported (Gauthier et al., 2000, Nat. Neurosci., 3:191-197) in a functional magnetic resonance imaging (fMRI) study that objects of visual expertise (cars and birds) activate the right fusiform face area (FFA) more strongly than non-expertise stimuli, and it was argued that the right FFA is involved in expertise specific rather than face specific visual processing. This expertise effect, however, may be due to experts taking advantage of the 'faceness' of the stimuli: birds have faces and three-quarter frontal views of cars resemble faces. This expertise effect may also be caused by a biased attentional modulation: with a blocked fMRI design, experts may attend more to a block of expertise than a block of non-expertise stimuli. In this study, using both side-view car images that do not resemble faces and bird images in an event-related fMRI design that minimizes attentional modulation, an expertise effect in the right FFA is observed in both car and bird experts (although a baseline bias makes the bird expertise effect less reliable). These results are consistent with those of Gauthier et al., and suggest the involvement of the right FFA in processing non-face expertise visual stimuli.

To efficiently extract visual information from complex visual scenes to guide behavior and thought, visual input needs to be organized into discrete units that can be selectively attended and processed. One important such selection unit is visual objects. A crucial factor determining object-based selection is the grouping between visual elements. Although human lesion data have pointed to the importance of the parietal cortex in object-based representations, our understanding of these parietal mechanisms in normal human observers remains largely incomplete. Here we show that grouped shapes elicited lower functional MRI (fMRI) responses than ungrouped shapes in inferior intraparietal sulcus (IPS) even when grouping was task-irrelevant. This relative ease of representing grouped shapes allowed more shape information to be passed onto later stages of visual processing, such as information storage in superior IPS, and may explain why grouped visual elements are easier to perceive than ungrouped ones after parietal brain lesions. These results are discussed within a neural object file framework, which argues for distinctive neural mechanisms supporting object individuation and identification in visual perception.brain imaging ͉ object processing ͉ visual attention ͉ working memory ͉ fMRI T o comprehend the continuous influx of visual information in everyday life, visual input needs to be organized into discrete units that can be selectively attended and processed. One important selection unit is visual objects, whose formation has been shown to profoundly impact subsequent visual processing (1). An important factor determining object-based selection is the grouping between visual elements by various gestalt principles (2, 3), such as connectedness and closure (4-7). Such grouping cues shape conscious visual perception. For example, after unilateral parietal lesions, observers' ability to perceive the presence of two objects, one on each side of the space, was greatly improved by connecting the two objects with a bar forming one big object with two parts instead of two separated objects (8-10). Likewise, after bilateral parietal lesions that result in Balint's syndrome (11), patients could still perceive a single complex object, but their ability to perceive the presence of multiple visual objects is severely impaired (11-13). Such lesion data point to the importance of the parietal cortex in object-based representations, but our understanding of these parietal object grouping and selection mechanisms in normal observers remains largely incomplete. Parietal brain responses have been associated with the number of visual objects actively represented in the mind, including those from the inferior intraparietal sulcus (IPS), which participates in attention-related processing (14-16), and the superior IPS, whose response correlates with the number of objects successfully stored in visual short-term memory (VSTM) (17-20, †). For example, when observers retain variable numbers of object shapes in VSTM, inferior IPS functiona...