Working memory is capacity-limited. In everyday life we rarely notice this limitation, in part because we develop behavioral strategies that help mitigate the capacity limitation. How behavioral strategies are mediated at the neural level is unclear, but a likely locus is lateral prefrontal cortex (LPFC). Neurons in LPFC play a prominent role in working memory and have been shown to encode behavioral strategies. To examine the role of LPFC in overcoming working-memory limitations, we recorded the activity of LPFC neurons in animals trained to perform a serial self-ordered search task. This task measured the ability to prospectively plan the selection of unchosen spatial search targets while retrospectively tracking which targets were previously visited. We found that individual LPFC neurons encoded the spatial location of the current search target but also encoded the spatial location of targets up to several steps away in the search sequence. Neurons were more likely to encode prospective than retrospective targets. When subjects used a behavioral strategy of stereotyped target selection, mitigating the working-memory requirements of the task, not only did the number of selection errors decrease but there was a significant reduction in the strength of spatial encoding in LFPC. These results show that LPFC neurons have spatiotemporal mnemonic fields, in that their firing rates are modulated both by the spatial location of future selection behaviors and the temporal organization of that behavior. Furthermore, the strength of this tuning can be dynamically modulated by the demands of the task.
Reinforcement learning models have proven highly effective for understanding learning in both artificial and biological systems. However, these models have difficulty in scaling up to the complexity of real-life environments. One solution is to incorporate the hierarchical structure of behavior. In hierarchical reinforcement learning, primitive actions are chunked together into more temporally abstract actions, called "options," that are reinforced by attaining a subgoal. These subgoals are capable of generating pseudoreward prediction errors, which are distinct from reward prediction errors that are associated with the final goal of the behavior. Studies in humans have shown that pseudoreward prediction errors positively correlate with activation of ACC. To determine how pseudoreward prediction errors are encoded at the single neuron level, we trained two animals to perform a primate version of the task used to generate these errors in humans. We recorded the electrical activity of neurons in ACC during performance of this task, as well as neurons in lateral prefrontal cortex and OFC. We found that the firing rate of a small population of neurons encoded pseudoreward prediction errors, and these neurons were restricted to ACC. Our results provide support for the idea that ACC may play an important role in encoding subgoals and pseudoreward prediction errors to support hierarchical reinforcement learning. One caveat is that neurons encoding pseudoreward prediction errors were relatively few in number, especially in comparison to neurons that encoded information about the main goal of the task.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.