Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read-write memory.

No abstract

The fields of neuroscience and artificial intelligence (AI) have a long and intertwined history. In more recent times, however, communication and collaboration between the two fields has become less commonplace. In this article, we argue that better understanding biological brains could play a vital role in building intelligent machines. We survey historical interactions between the AI and neuroscience fields and emphasize current advances in AI that have been inspired by the study of neural computation in humans and other animals. We conclude by highlighting shared themes that may be key for advancing future research in both fields.

Stimulus-evoked neural activity is attenuated upon stimulus repetition ('repetition suppression'), a phenomenon attributed to largely automatic processes in sensory neurons. By manipulating the likelihood of stimulus repetition, we show that repetition suppression in the human brain is reduced when stimulus repetitions are improbable (and thus, unexpected). These data suggest that repetition suppression reflects a relative reduction in top-down perceptual 'prediction error' when processing an expected compared to an unexpected stimulus.Stimulus-specific repetition suppression (RS) -the relative attenuation in neural signal evoked by the repeated occurrence of a stimulus -is among the best-known neural phenomena [1][2][3][4] , and has been widely employed in functional magnetic resonance imaging (fMRI) studies to define functional properties of brain regions 5,6 and explore neural substrates of behavioral priming effects 2,4 . However, the neurocomputational basis for RS remains controversial 1 . Two influential theories view RS as a relatively automatic consequence of the bottom-up flow of perceptual information through sensory cortex: either neurons tuned to the repeated stimulus fatigue 1 , or subsequent presentations of a stimulus are encoded more sparsely (and efficiently), leading to a sharpening in the population of neurons recruited 4,7 . By contrast, a recent model of perceptual inference casts RS as a consequence of top-down perceptual expectations 2,8 : here, RS reflects a reduction in perceptual 'prediction error' (the neural signal evoked by a mismatch between expected and observed percepts) that occurs when sensory evidence conforms to a more probable (previously seen) compared to a less probable (novel) percept. Unlike other theories, the prediction error model holds that RS will vary with contextual factors that affect subjects' perceptual expectations, and suggests that RS will be reduced under conditions where stimulus repetitions are unexpected.We created such a situation by presenting subjects (n = 16), who had provided informed written consent, on each trial with either the same face twice, or two different faces, in two experimental contexts -one where repetitions occurred more frequently than alternations, and one where the reverse was the case. Importantly, all face exemplars were trial-unique, such that the NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript probability of a repetition per se, and not the frequency of repetition of a specific face, varied between blocks ( Fig. 1a and Supplementary Methods online). Incidental to this manipulation, subjects were required to make a speeded response to occasional inverted faces ('targets') 9 . Limiting our analysis to non-target trials, we measured how face-sensitive visual cortex responded to face repetitions ('rep trials') and face alternations ('alt trials') that were either expected (in 'REP BLOCKS') or unexpected (in 'ALT BLOCKS'), comparing these estimates in 2 × 2 factorial mixed block/event-related desi...

No abstract

Sensory signals are highly structured in both space and time. These structural regularities in visual information allow expectations to form about future stimulation, thereby facilitating decisions about visual features and objects. Here, we discuss how expectation modulates neural signals and behaviour in humans and other primates. We consider how expectations bias visual activity before a stimulus occurs, and how neural signals elicited by expected and unexpected stimuli differ. We discuss how expectations may influence decision signals at the computational level. Finally, we consider the relationship between visual expectation and related concepts, such as attention and adaptation.

People are capable of robust evaluations of their decisions: they are often aware of their mistakes even without explicit feedback, and report levels of confidence in their decisions that correlate with objective performance. These metacognitive abilities help people to avoid making the same mistakes twice, and to avoid overcommitting time or resources to decisions that are based on unreliable evidence. In this review, we consider progress in characterizing the neural and mechanistic basis of these related aspects of metacognition—confidence judgements and error monitoring—and identify crucial points of convergence between methods and theories in the two fields. This convergence suggests that common principles govern metacognitive judgements of confidence and accuracy; in particular, a shared reliance on post-decisional processing within the systems responsible for the initial decision. However, research in both fields has focused rather narrowly on simple, discrete decisions—reflecting the correspondingly restricted focus of current models of the decision process itself—raising doubts about the degree to which discovered principles will scale up to explain metacognitive evaluation of real-world decisions and actions that are fluid, temporally extended, and embedded in the broader context of evolving behavioural goals.

Incoming sensory information is often ambiguous, and the brain has to make decisions during perception. "Predictive coding" proposes that the brain resolves perceptual ambiguity by anticipating the forthcoming sensory environment, generating a template against which to match observed sensory evidence. We observed a neural representation of predicted perception in the medial frontal cortex, while human subjects decided whether visual objects were faces or not. Moreover, perceptual decisions about faces were associated with an increase in top-down connectivity from the frontal cortex to face-sensitive visual areas, consistent with the matching of predicted and observed evidence for the presence of faces.