Our choice is influenced by choices we made in the past, but the mechanism responsible for the choice bias remains elusive. Here we show that the history-dependent choice bias can be explained by an autonomous learning rule whereby an estimate of the likelihood of a choice to be made is updated in each trial by comparing between the actual and expected choices. We found that in perceptual decision making without performance feedback, a decision on an ambiguous stimulus is repeated on the subsequent trial more often than a decision on a salient stimulus. This inertia of decision was not accounted for by biases in motor response, sensory processing, or attention. The posterior cingulate cortex and frontal eye field represent choice prediction error and choice estimate in the learning algorithm, respectively. Interactions between the two regions during the intertrial interval are associated with decision inertia on a subsequent trial.

Our voluntary behaviors are thought to be controlled by top-down signals from the prefrontal cortex that modulate neural processing in the posterior cortices according to the behavioral goal. However, we have insufficient evidence for the causal effect of the top-down signals. We applied a single-pulse transcranial magnetic stimulation over the human prefrontal cortex and measured the strength of the top-down signals as an increase in the efficiency of neural impulse transmission. The impulse induced by the stimulation transmitted to different posterior visual areas depending on the domain of visual features to which subjects attended. We also found that the amount of impulse transmission was associated with the level of attentional preparation and the performance of visual selective-attention tasks, consistent with the causal role of prefrontal top-down signals.

In many natural environments the value of a choice gradually gets better or worse as circumstances change. Discerning such trends makes predicting future choice values possible. We show that humans track such trends by comparing estimates of recent and past reward rates, which they are able to hold simultaneously in the dorsal anterior cingulate cortex (dACC). Comparison of recent and past reward rates with positive and negative decision weights is reflected by opposing dACC signals indexing these quantities. The relative strengths of time-linked reward representations in dACC predict whether subjects persist in their current behaviour or switch to an alternative. Computationally, trend-guided choice can be modelled by using a reinforcement-learning mechanism that computes a longer-term estimate (or expectation) of prediction errors. Using such a model, we find a relative predominance of expected prediction errors in dACC, instantaneous prediction errors in the ventral striatum and choice signals in the ventromedial prefrontal cortex.

In complex environments, many potential cues can guide a decision or be assigned responsibility for the outcome of the decision. We know little, however, about how humans and animals select relevant information sources that should guide behavior. We show that subjects solve this relevance selection and credit assignment problem by selecting one cue and its association with a particular outcome as the main focus of a hypothesis. To do this, we examined learning while using a task design that allowed us to estimate the focus of each subject's hypotheses on a trial-by-trial basis. When a prediction is confirmed by the outcome, then credit for the outcome is assigned to that cue rather than an alternative. Activity in medial frontal cortex is associated with the assignment of credit to the cue that is the main focus of the hypothesis. However, when the outcome disconfirms a prediction, the focus shifts between cues, and the credit for the outcome is assigned to an alternative cue. This process of reselection for credit assignment to an alternative cue is associated with lateral orbitofrontal cortex.

Executive control refers to the regulation of cognition and behavior by mental processes and is a hallmark of higher cognition. Most approaches to understanding its mechanisms begin with the assumption that our brains have anatomically segregated and functionally specialized control modules. The modular approach is intuitive: Control is conceptually distinct from basic mental processing, so an organization that reifies that distinction makes sense. An alternative approach sees executive control as self-organizing principles of a distributed organization. In distributed systems, control and controlled processes are colocalized within large numbers of dispersed computational agents. Control then is often an emergent consequence of simple rules governing the interaction between agents. Because these systems are unfamiliar and unintuitive, here we review several well-understood examples of distributed control systems, group living insects and social animals, and emphasize their parallels with neural systems. We then reexamine the cognitive neuroscience literature on executive control for evidence that its neural control systems may be distributed.

Humans tend to avoid mental effort. Previous studies have demonstrated this tendency using various demand-selection tasks; participants generally avoid options associated with higher cognitive demand. However, it remains unclear whether humans avoid mental effort adaptively in uncertain and non-stationary environments, and if so, what neural mechanisms underlie this learned avoidance and whether they remain the same irrespective of cognitive-demand types. We addressed these issues by developing novel demand-selection tasks where associations between choice options and cognitive-demand levels change over time, with two variations using mental arithmetic and spatial reasoning problems (29:4 and 18:2 males:females). Most participants showed avoidance, and their choices depended on the demand experienced on multiple preceding trials. We assumed that participants updated the expected cost of mental effort through experience, and fitted their choices by reinforcement learning models, comparing several possibilities. Model-based fMRI analyses revealed that activity in the dorsomedial and lateral frontal cortices was positively correlated with the trial-by-trial expected cost for the chosen option commonly across the different types of cognitive demand, and also revealed a trend of negative correlation in the ventromedial prefrontal cortex. We further identified correlates of cost-prediction-error at time of problem-presentation or answering the problem, the latter of which partially overlapped with or were proximal to the correlates of expected cost at time of choice-cue in the dorsomedial frontal cortex. These results suggest that humans adaptively learn to avoid mental effort, having neural mechanisms to represent expected cost and cost-prediction-error, and the same mechanisms operate for various types of cognitive demand.In daily life, humans encounter various cognitive demands, and tend to avoid high-demand options. However, it remains unclear whether humans avoid mental effort adaptively under dynamically changing environments, and if so, what are the underlying neural mechanisms and whether they operate irrespective of cognitive-demand types. To address these issues, we developed novel tasks, where participants could learn to avoid high-demand options under uncertain and non-stationary environments. Through model-based fMRI analyses, we found regions whose activity was correlated with the expected mental effort cost, or cost-prediction-error, regardless of demand-type, with overlap or adjacence in the dorsomedial frontal cortex. This finding contributes to clarifying the mechanisms for cognitive-demand avoidance, and provides empirical building blocks for the emerging computational theory of mental effort.

Our ability to choose nonhabitual controlled behavior instead of habitual automatic behavior is based on a flexible control mechanism subserved by neural activity representing the behavior-guiding rule. However, it has been shown that the behavior slows down more when switching from controlled to automatic behavior than vice versa. Here we show that persistent effective connectivity of the neural network after execution of controlled behavior is responsible for the behavioral slowing on a subsequent trial. We asked normal human subjects to perform a prosaccade or antisaccade task based on a cue and examined the effective connectivity of the neural network based on the pattern of neural impulse transmission induced by stimulation of the frontal eye field (FEF). Effective connectivity during the task preparation period was dependent on the task that subjects had performed on the previous trial, regardless of the upcoming task. The strength of this persistent effective connectivity was associated with saccade slowing especially on trials after controlled antisaccade. In contrast, the pattern of regional activation changed depending on the upcoming task regardless of the previous task and the decrease in activation was associated with errors in upcoming antisaccade task. These results suggest that the effective connectivity examined by FEF stimulation reflects a residual functional state of the network involved in performance of controlled antisaccade and its persistence may account for the behavioral slowing on the subsequent trial.

Biological and artificial intelligence (AI) are often defined by their capacity to achieve a hierarchy of short-term and long-term goals that require incorporating information over time and space at both local and global scales. More advanced forms of this capacity involve the adaptive modulation of integration across scales, which resolve computational inefficiency and explore-exploit dilemmas at the same time. Research in neuroscience and AI have both made progress towards understanding architectures that achieve this. Insight into biological computations come from phenomena such as decision inertia, habit formation, information search, risky choices and foraging. Across these domains, the brain is equipped with mechanisms (such as the dorsal anterior cingulate and dorsolateral prefrontal cortex) that can represent and modulate across scales, both with top-down control processes and by local to global consolidation as information progresses from sensory to prefrontal areas. Paralleling these biological architectures, progress in AI is marked by innovations in dynamic multiscale modulation, moving from recurrent and convolutional neural networks—with fixed scalings—to attention, transformers, dynamic convolutions, and consciousness priors—which modulate scale to input and increase scale breadth. The use and development of these multiscale innovations in robotic agents, game AI, and natural language processing (NLP) are pushing the boundaries of AI achievements. By juxtaposing biological and artificial intelligence, the present work underscores the critical importance of multiscale processing to general intelligence, as well as highlighting innovations and differences between the future of biological and artificial intelligence.