A Probability Distribution over Latent Causes, in the Orbitofrontal Cortex

Abstract

The orbitofrontal cortex (OFC) has been implicated in both the representation of "state," in studies of reinforcement learning and decision making, and also in the representation of "schemas," in studies of episodic memory. Both of these cognitive constructs require a similar inference about the underlying situation or "latent cause" that generates our observations at any given time. The statistically optimal solution to this inference problem is to use Bayes' rule to compute a posterior probability distribution over latent causes. To test whether such a posterior probability distribution is represented in the OFC, we tasked human participants with inferring a probability distribution over four possible latent causes, based on their observations. Using fMRI pattern similarity analyses, we found that BOLD activity in the OFC is best explained as representing the (log-transformed) posterior distribution over latent causes. Furthermore, this pattern explained OFC activity better than other task-relevant alternatives, such as the most probable latent cause, the most recent observation, or the uncertainty over latent causes.Key words: Bayes' rule; context; posterior distribution; schemas; state representation; ventromedial prefrontal cortex IntroductionIn recent years, cognitive neuroscientists studying reinforcement learning and decision making have recognized the importance of specifying representations of the environmental "state" that capture the structure of the world in a predictive way (Courville et al., 2004;Gershman and Niv, 2010). At the same time, there has been renewed interest among cognitive neuroscientists in how memory encoding and retrieval are shaped by situation-specific prior knowledge ("schemas";Tse et al., 2007). As work in this area progresses, it is important to clarify exactly what constitutes a schema and how schemas are formed.Whether inferring the current state or the currently relevant schema, agents are making inferences about the hidden variables that underlie and generate our observations in the world. This inference can be concretely formulated in terms of Bayesian latent cause models (e.g., Gershman et al., 2010). According to this framework, states and schemas can be viewed as hidden (latent) causes that give rise to observable events. For example, if you arrive late to a lecture, the situation (whether this is indeed the department colloquium or you have accidentally walked in on an undergraduate class) determines your observations about the average age of the audience, the proportion of audience members that are taking notes, the type of information being presented, and so on. To decide whether you are in the right place, you can use Bayesian inference to infer a belief distribution over the possible situations that might have generated the current observations, i.e., a posterior probability distribution over latent causes, p(latent cause ͉ observations) (Fig. 1A).We hypothesized, based on the similarity of the underlying computations, that the inference related to these two cogni...

Lateral orbitofrontal cortex anticipates choices and integrates prior with current information

“…This result suggests that the neuronal representation of the state-space interacts in the OFC with the decision-making process, potentially by facilitating the combination of prior with current information. This result is consistent with a recent human fMRI study suggesting that OFC represents posterior probability distributions by integrating extended prior experience with current information58.…”

Does mental context drift or shift?

“…Also, if multiple latent causes are active, how do we assign credit so that we update the appropriate latent cause when new information comes in? These issues intersect with the existing literatures on cognitive control, multitasking, and reinforcement learning [44*,52,63,64]. Drawing on these literatures might provide useful insight into how latent-cause models could be implemented in the brain, and how they interact with neural memory processes.…”

“…By contrast, when changes are small and predictable, a single latent cause is inferred and memories are more malleable and susceptible to interference [49,50]. Neurally, the orbitofrontal cortex, which is sensitive to unobservable states [51], has been shown to represent the probability that each latent cause is currently active during the inference process [52]. Thus, one intriguing possibility is that the orbitofrontal cortex may draw on memory representations from the hippocampus and temporally extended situational information from the PMN to represent the current context for goals and actions.…”

Memory integration constructs maps of space, time, and concepts

“…When presented with a stimulus to be assigned to a concept, hippocampus—mPFC interactions may trigger reactivation of similar, concept-relevant learning experiences. Medial PFC also influences hippocampal encoding by compressing memory representations to emphasize features that capture commonalities across events [21], consistent with its hypothesized role in forming mental models of latent structures that are not directly observable [13, 14, 68]. The retrieved memories may then be used to predict a concept label for the current experience.…”

“…Reactivation of related memories during new events is mediated by hippocampal pattern completion processes that allow for reinstatement of entire memory traces from overlapping input [11, 12]. Medial PFC, which is thought to represent mental models that guide behavior [13, 14], may further bias hippocampal pattern completion to the most relevant prior knowledge [9]. Hippocampus then signals deviations between current events and reactivated memories, triggering memory updating [1, 15, 16].…”