Brains use predictive models to facilitate the processing of expected stimuli or planned actions. Under a predictive model, surprising (low probability) stimuli or actions necessitate the immediate reallocation of processing resources, but they can also signal the need to update the underlying predictive model to reflect changes in the environment. Surprise and updating are often correlated in experimental paradigms but are, in fact, distinct constructs that can be formally defined as the Shannon information (I S ) and KullbackLeibler divergence (D KL ) associated with an observation. In a saccadic planning task, we observed that distinct behaviors and brain regions are associated with surprise/I S and updating/D KL . Although surprise/I S was associated with behavioral reprogramming as indexed by slower reaction times, as well as with activity in the posterior parietal cortex [human lateral intraparietal area (LIP)], the anterior cingulate cortex (ACC) was specifically activated during updating of the predictive model (D KL ). A second saccade-sensitive region in the inferior posterior parietal cortex (human 7a), which has connections to both LIP and ACC, was activated by surprise and modulated by updating. Pupillometry revealed a further dissociation between surprise and updating with an early positive effect of surprise and late negative effect of updating on pupil area. These results give a computational account of the roles of the ACC and two parietal saccade regions, LIP and 7a, by which their involvement in diverse tasks can be understood mechanistically. The dissociation of functional roles between regions within the reorienting/reprogramming network may also inform models of neurological phenomena, such as extinction and Balint syndrome, and neglect.eye movement | prediction | attention | learning | Bayes I n a nonrandom environment, brains can and should make use of past experience to facilitate the processing of incoming sensory information and the selection of actions, through prediction (1, 2). An important aspect of brain function is therefore the construction and tuning of internal models to represent statistics of the environment that are relevant for future behavior.The use of predictive internal models implies that not only are some events well predicted (high probability under the model) but, conversely, some events (which have a low probability under the model) are surprising (3). Surprising events may be associated with behavioral costs; for example, although valid attentional cues speed reaction times (RTs), invalid cues lengthen them (3, 4). However, surprising events can have a further significance for the observer in that they sometimes provide evidence for a change in the environment, which would imply a need to update the brain's internal models to predict future events accurately.Here, we explore the possibility that the brain carries out at least two distinct operations when a surprising event occurs: (i) within trial reorienting processes evoked by surprise, including reallocation o...

Four experiments were conducted with rats in order to determine whether being placed on a platform in one corner of a rectangular swimming pool results in latent spatial learning. Rats in Experiments 1–3 received four trials a day of being placed on the platform. During a subsequent test trial, in which they were released into the pool without the platform, the rats exhibited a preference for swimming in the correct corners of the pool (those with the same geometric properties as the corner containing the platform during training), than the two remaining, incorrect corners. This effect was seen when the interval between the final placement trial and the test trial was as much as 24 hr (Experiment 2) and after varying numbers of sessions of placement training (Experiment 3). Experiment 4 revealed that when the test took place in a kite-shaped arena, after placement training in a rectangle, a stronger preference was shown for the corner that was geometrically equivalent to the correct rather than the incorrect corners in the rectangle. The placement treatment is said to result in latent spatial learning based on the development of S-S associations.

One of the most influential accounts of central orbitofrontal cortex-that it mediates behavioral flexibility-has been challenged by the finding that discrimination reversal in macaques, the classic test of behavioral flexibility, is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests that the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity, a group of nine macaques were taught discrimination reversal learning tasks, and its impact on gray matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups, each comprising 10 animals. One control group learned discrimination tasks that were similar but lacked any reversal component, and the other control group engaged in no learning. Gray matter changes were prominent in posterior orbitofrontal cortex/anterior insula but were also found in three other frontal cortical regions: lateral orbitofrontal cortex (orbital part of area 12 [12o]), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest, and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison with controls. In a final set of experiments, we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination a1111111111 a1111111111 a1111111111 a1111111111 a1111111111

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether that information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scare resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether that information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback, about the option not chosen. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

Rats were trained in two experiments to find a submerged platform that was situated in one of two of the four corners of a rectangular pool with a curved long wall. Different landmarks occupied two of the corners on every trial and the platform was always situated near a landmark. For the place group in each experiment, the location of the platform was indicated by the shape of the pool and stimuli outside the pool (place cues), but not the landmarks within the pool. For the landmark groups, the landmarks indicated where the platform could be found, but not the place cues. During Stage 2, two of the place cues were relevant, and two of the landmarks were irrelevant, for a new discrimination. The place cues gained better control over searching for the platform in the place than the landmark group when the place cues had initially been relevant by signalling the presence (Experiment 1) or the absence (Experiment 2) of the platform. The results are said to show that animals pay more attention to relevant than irrelevant cues.Many of the discriminations made by animals involve some stimuli that are relevant to the solution, and some that are irrelevant. If the same stimuli are used for a new discrimination, then learning often progresses more readily with the previously relevant than irrelevant stimuli, (e.g. Lawrence, 1949;1950). Furthermore, if relevant and irrelevant stimuli are from different dimensions, and the new discrimination involves novel stimuli from these dimensions, learning often progresses more readily with the novel stimuli that belong to the previously relevant rather than the previously irrelevant dimensions (e.g. Mackintosh & Little, 1969). These differences in conditionability, or associability, have been attributed to animals paying more attention to relevant than irrelevant stimuli during discrimination learning (e.g. Sutherland & Mackintosh, 1971;Mackintosh, 1975). The purpose of the present paper is to explore the generality of the circumstances that promote these changes in attention.One source of support for the claim that animals pay more attention to relevant than to irrelevant stimuli is the intradimensional shift -extradimensional shift (IDS-EDS) effect. Typically, subjects receive a number of discriminations of the form AX+ BX−, AY+ BY−, where A and B represent values from one dimension, say colour, which is relevant to the solution of the discriminations and X and Y represent values from another, irrelevant dimension, say orientation. Once the initial discriminations have been mastered, subjects are transferred to a set of new discriminations involving novel values from the two dimensions.

The effect of US signalling and the US–CS interval in backward conditioning was assessed in mice. For one group of mice the presentation of food was signalled by a tone and for another group, food was unsignalled. For half of the mice, within each group, the presentation of food preceded a visual cue by 10 s. For the other half, food was presented at the start of the visual cue (0-s US–CS interval), resulting in simultaneous pairings of these events. A summation test and a subsequent retardation test were used to assess the inhibitory effects of backward conditioning in comparison to training with a non-reinforced visual cue that controlled for the possible effects of latent inhibition and conditioned inhibition caused as a consequence of differential conditioning. In the summation test unsignalled presentations of the US resulted in inhibition when the US–CS interval was 10 s, but not 0 s. Signalled presentations of the US resulted in inhibition, independent of the US–CS interval. In the retardation test, independent of US signalling, a US–CS interval of 10 s failed to result in inhibition, but an interval of 0 s resulted in greater conditioned responding to the backward CS than the control CS. A generalisation decrement account of the effect of signalling the US with a 0-s US–CS interval, which resulted in reduced responding in the summation test and faster acquisition in the retardation test, is discussed.

One of the most influential accounts of central orbitofrontal cortex– that it mediates behavioral flexibility – has been challenged by the finding that discrimination reversal in macaques –the classic test of behavioral flexibility –is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity a group of nine macaques were taught discrimination reversal learning tasks and its impact on grey matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups each comprising ten animals. One control group learned similar discrimination tasks but which lacked any reversal component and the other control group engaged in no learning. Grey matter changes were prominent in posterior orbitofrontal cortex/anterior insula but also were found in three other frontal cortical regions: lateral orbitofrontal cortex (12o), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison to controls. In a final set of experiments we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination learning, affected structure and activity in the same frontal cortical circuit. The results identify a distributed frontal cortical circuit associated with behavioral flexibility.