Decoding the future from past experience: learning shapes predictions in early visual cortex

Abstract

Learning the structure of the environment is critical for interpreting the current scene and predicting upcoming events. However, the brain mechanisms that support our ability to translate knowledge about scene statistics to sensory predictions remain largely unknown. Here we provide evidence that learning of temporal regularities shapes representations in early visual cortex that relate to our ability to predict sensory events. We tested the participants' ability to predict the orientation of a test stimulus after exposure to sequences of leftward- or rightward-oriented gratings. Using fMRI decoding, we identified brain patterns related to the observers' visual predictions rather than stimulus-driven activity. Decoding of predicted orientations following structured sequences was enhanced after training, while decoding of cued orientations following exposure to random sequences did not change. These predictive representations appear to be driven by the same large-scale neural populations that encode actual stimulus orientation and to be specific to the learned sequence structure. Thus our findings provide evidence that learning temporal structures supports our ability to predict future events by reactivating selective sensory representations as early as in primary visual cortex.

Not All Predictions Are Equal: “What” and “When” Predictions Modulate Activity in Auditory Cortex through Different Mechanisms

“…Converging evidence suggests that this modulation has motor, rather than sensory, origins, especially in rhythmic contexts (Schubotz et al, 2000;Cravo et al, 2011;Morillon et al, 2015). Dopaminergic manipulations affect timing and "when" predictability, partly relying on the nigrostriatal motor pathway (Coull et al, 2012;Narayanan et al, 2012;Parker et al, 2013), consistent with the dual-stream hypothesis and the involvement of motor regions in the coding of sound sequences (Leaver et al, 2009;Bornkessel-Schlesewsky et al, 2015). Because such classical neuromodulatory effects are not necessarily voltage-dependent (Formenti et al, 1998;Gorelova et al, 2002), we hypothesized that "when" predictability might be mediated by activity-independent gain modulation (whereby the gain of principal cells is not directly modulated by descending cortical inputs, but regulated by putative classical neuromodulators, and thus prone to more distal and subcortical influences) expressed in motor and/or sensory regions.…”

“…Activity-independent gain modulation, on the other hand, translates into disinhibition of a given region without additional weighting of the strength of this disinhibition by inputs from other regions. We focused on those three cortical regions, given the often conflicting evidence for the effects of predictability at the level of sensory (Griffin et al, 2002;den Ouden et al, 2009;Alink et al, 2010;Turk-Browne et al, 2010;Arnal and Giraud, 2012;Lakatos et al, 2013;Summerfield and de Lange, 2014;Luft et al, 2015), motor (Morillon et al, 2015, and FC, including the inferior and middle frontal gyri (den Ouden et al, 2009;Turk-Browne et al, 2010;Coull et al, 2011); and also because our electrophysiological results showed modulations of those areas as a function of "what" and "when" predictions, a prerequisite for DCM.…”

“…"When" predictability has also been linked to sensory gain modulation (Schroeder and Lakatos, 2009;Rohenkohl et al, 2012). Converging evidence suggests that this modulation has motor, rather than sensory, origins, especially in rhythmic contexts (Schubotz et al, 2000;Cravo et al, 2011;Morillon et al, 2015).…”

The hippocampus: a special place for time

“…There is strong evidence to suggest that episodes are encoded as organized sequences of events, [5][6][7] such that retrieving one event obligatorily triggers recollection, 8,9 or implicit retrieval, [10][11][12] of the rest of the sequence. In addition to facil-itating memory for the past, temporal sequence representation allows people to make predictions about the future, [13][14][15][16] and it is fundamental for spatial cognition, 17,18 narrative comprehension, 19,20 and imagination and mental simulation. 21,22 Despite the centrality of time to episodic memory and cognition, until recently little was known about the neural mechanisms that support temporal organization in memory.…”

Low beta power reflects perceived temporal probabilities for uncertain future events

“…The hazard rate reflects how the brain uses elapsed time to dynamically update the expectations for a future deterministic target to occur (Janssen and Shadlen, 2005;Leon and Shadlen, 2003). In principle, participants could simply estimate the probability density function of a target onset over trials, and use this information to respond (Luft et al, 2015). However, "that is not the natural way one thinks about it as the [waiting] process unfolds in time.…”