One of the most fundamental functions of the brain is to predict upcoming events on the basis of the recent past. A closely related function is to signal when a prediction has been violated. The identity of the brain regions that mediate these functions is not known. We set out to determine whether they are implemented at the level of single neurons in the visual system. We gave monkeys prolonged exposure to pairs of images presented in fixed sequence so that each leading image became a strong predictor for the corresponding trailing image. We then monitored the responses of neurons in the inferotemporal cortex to image sequences that obeyed or violated the transitional rules imposed during training. Inferotemporal neurons exhibited a transitional surprise effect, responding much more strongly to unpredicted transitions than to predicted transitions. Thus, neurons even in the visual system make experience-based predictions and react when they fail.macaque | vision | plasticity T he inferotemporal cortex (ITC), the terminus of the ventral stream of visual areas (1), plays a critical role in object vision (2, 3). ITC neurons respond with individual patterns of selectivity to complex images (4). Training monkeys to discriminate between images (5-7), categorize them (8, 9), or form associations between them (10-12) induces functional changes among neurons in the ITC which have the effect of strengthening the representation of image attributes relevant to task performance. Even passive viewing causes changes in neuronal visual responsiveness. Repeated viewing of a single image leads to a weakening of responses to it (13,14). Repeated viewing of two images close together in time leads to pair coding: a tendency for neurons responsive to one image also to respond to the other (15-17). The effects of passive viewing, because they do not depend on task demands, fall into the category of unsupervised statistical learning.An important form of unsupervised learning not previously studied at the level of single neurons concerns transitional statistics. The learning of transitional statistics has been the focus of much behavioral study in humans because it is thought to underlie the development during infancy of the ability to perceive event boundaries including word boundaries in speech (18,19). Human infants passively exposed to a stimulus stream in which certain visual images always follow certain others automatically register the transitional rules as evidenced by their orienting preferentially to a test stream containing novel transitions (20). The adult human brain is sensitive to transitional probabilities, as evidenced by its generating strong responses to improbable transitions at the level of scalp potential and blood oxygenation measures (21-27). Monkeys, like human infants, have been reported to learn transitional probabilities and to orient preferentially to improbable transitions (28). No effort has been made as yet to characterize the underlying neuronal mechanisms (29). We hypothesized that neurons in the ITC ...

The prefrontal cortex is known to represent different types of information in working memory. Contrasting theories propose that the dorsal and ventral regions of the lateral prefrontal cortex are innately specialized for the representation of spatial and non-spatial information respectively (Goldman-Rakic, 1996), or that the two regions are shaped by the demands of cognitive tasks imposed on them (Miller, 2000). To resolve this issue, we recorded from neurons in the two regions, prior to and at multiple stages of training monkeys on visual working memory tasks. Prior to training, substantial functional differences were present between the two regions. Dorsal prefrontal cortex exhibited higher overall responsiveness to visual stimuli and higher selectivity for spatial information. After training, stimulus selectivity generally decreased, though dorsal prefrontal cortex retained higher spatial selectivity regardless of task performed. Ventral prefrontal cortex appeared to be affected to a greater extent by the nature of task performed. Our results indicate that regional specialization for stimulus selectivity is present in the primate prefrontal cortex regardless of training. Dorsal areas of the prefrontal cortex are inherently organized to represent spatial information and training has little influence on this spatial bias. Ventral areas are biased toward non-spatial information although they are more influenced by training both in terms of activation and changes in stimulus selectivity.

The prefrontal cortex is considered essential for learning to perform cognitive tasks though little is known about how the representation of stimulus properties is altered by learning. To address this issue, we recorded neuronal activity in monkeys before and after training on a task that required visual working memory. After the subjects learned to perform the task, we observed activation of more prefrontal neurons and increased activity during working memory maintenance. The working memory-related increase in firing rate was due mostly to regular-spiking putative pyramidal neurons. Unexpectedly, the selectivity of neurons for stimulus properties and the ability of neurons to discriminate between stimuli decreased as the information about stimulus properties was apparently present in neural firing prior to training and neuronal selectivity degraded after training in the task. The effect was robust and could not be accounted for by differences in sampling sites, selection of neurons, level of performance, or merely the elapse of time. The results indicate that, in contrast to the effects of perceptual learning, mastery of a cognitive task degrades the apparent stimulus selectivity as neurons represent more abstract information related to the task. This effect is countered by the recruitment of more neurons after training.

Our visual memory percepts of whether we have encountered specific objects or scenes before are hypothesized to manifest as decrements in neural responses in inferotemporal cortex (IT) with stimulus repetition. To evaluate this proposal, we recorded IT neural responses as two monkeys performed a single-exposure visual memory task designed to measure the rates of forgetting with time. We found that a weighted linear read-out of IT was a better predictor of the monkeys’ forgetting rates and reaction time patterns than a strict instantiation of the repetition suppression hypothesis, expressed as a total spike count scheme. Behavioral predictions could be attributed to visual memory signals that were reflected as repetition suppression and were intermingled with visual selectivity, but only when combined across the most sensitive neurons.

Repeated viewing of an image over days and weeks induces a marked reduction in the strength with which neurons in monkey inferotemporal cortex respond to it. The processing advantage that attaches to this reduction is unknown. One possibility is that truncation of the response to a familiar image leaves neurons in a state of readiness to respond to ensuing images and thus enhances their ability to track rapidly changing displays. We have explored this possibility by assessing neuronal responses to familiar and novel images in rapid serial visual displays. Inferotemporal neurons respond more strongly to familiar than to novel images in such displays. The effect is stronger among putative inhibitory neurons than among putative excitatory neurons. A comparable effect occurs at the level of the scalp potential in humans. We conclude that long-term familiarization sharpens the response dynamics of neurons in both monkey and human extrastriate visual cortex.

If monkeys repeatedly, over the course of weeks, view displays in which two images appear in fixed sequence, then neurons of inferotemporal cortex (ITC) come to exhibit prediction suppression. The response to the trailing image is weaker if it follows the leading image with which it was paired during training than if it follows some other leading image. Prediction suppression is a plausible neural mechanism for statistical learning of visual transitions such as has been demonstrated in behavioral studies of human infants and adults. However, in the human studies, subjects are exposed to continuous sequences in which the same image can be both predicted and predicting and statistical dependency can exist between nonadjacent items. The aim of the present study was to investigate whether prediction suppression in ITC develops under such circumstances. To resolve this issue, we exposed monkeys repeatedly to triplets of images presented in fixed order. The results indicate that prediction suppression can be induced by training not only with pairs of images but also with longer sequences.

Most accounts of image and object encoding in inferotemporal cortex (IT) focus on the distinct patterns of spikes that different images evoke across the IT population. By analyzing data collected from IT as monkeys performed a visual memory task, we demonstrate that variation in a complementary coding scheme, the magnitude of the population response, can largely account for how well images will be remembered. To investigate the origin of IT image memorability modulation, we probed convolutional neural network models trained to categorize objects. We found that, like the brain, different natural images evoked different magnitude responses from these networks, and in higher layers, larger magnitude responses were correlated with the images that humans and monkeys find most memorable. Together, these results suggest that variation in IT population response magnitude is a natural consequence of the optimizations required for visual processing, and that this variation has consequences for visual memory.

Neurons in the prefrontal cortex and a network of interconnected brain areas discharge in a persistent fashion after the offset of sensory stimulation. Such persistent discharges are thought to constitute a neuronal correlate of working memory. The information content of neuronal discharges and its anatomical localization across the surface of the prefrontal cortex has been a matter of debate. Discrepant results by different laboratories may be due to the effects of different training regiments and tasks used in memory tasks. In order to address how training in a memory task alters neuronal responses, we performed recordings in monkeys that were never trained in memory tasks, but passively viewed visual stimuli. We have found that a population of prefrontal neurons responded to visual stimuli and also exhibited significantly elevated responses during "delay" intervals of the task. For a population of these neurons, persistent discharges were selective for the location and feature of the preceding stimulus. These discharges were typically disrupted by the appearance of a subsequent stimulus. Our results suggest that some prefrontal neurons represent the location and identity of visual stimuli in a persistent fashion, even when the latter are not behaviorally important or required to be kept in memory.