A Scale-Invariant Internal Representation of Time

Shankar, Karthik H.; Howard, Marc W.

doi:10.1162/neco_a_00212

Cited by 134 publications

(195 citation statements)

References 73 publications

Supporting

Mentioning

179

Contrasting

Order By: Relevance

“…Recent works have provided direct evidence for a representation of time in the striatum that is distributed over a set of neurons (49,50) and that DA neurons may directly modulate timing (47). The specific set of functions adopted in our work (43,51) is a possible realization of these findings. Although there are alternative temporal representations (45) and approaches (52), our choice was dictated for the sake of simplicity and because there exist detailed studies of this internal representation that make its use attractive (53).…”

Section: Discussionsupporting

confidence: 53%

See 1 more Smart Citation

Dopamine reward prediction error signal codes the temporal evaluation of a perceptual decision report

Sarno

Lafuente

Parga

2017

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Learning to associate unambiguous sensory cues with rewarded choices is known to be mediated by dopamine (DA) neurons. However, little is known about how these neurons behave when choices rely on uncertain reward-predicting stimuli. To study this issue we reanalyzed DA recordings from monkeys engaged in the detection of weak tactile stimuli delivered at random times and formulated a reinforcement learning model based on belief states. Specifically, we investigated how the firing activity of DA neurons should behave if they were coding the error in the prediction of the total future reward when animals made decisions relying on uncertain sensory and temporal information. Our results show that the same signal that codes for reward prediction errors also codes the animal's certainty about the presence of the stimulus and the temporal expectation of sensory cues.dopamine activity | perception | temporal expectation | decision making | reinforcement learning W hen an inexperienced animal hears a soft rustle in the nearby foliage, it does not associate this cue with the escaping prey that it observes immediately after. How does the animal get to learn that the correct action to take is to approach it and try to get it? In perceptual decision-making experiments, animals learn how to make decisions based on their perception of weak sensory stimuli, receiving a reward for their correct choices, which they are taught to communicate by means of a specific motor action (1-7). The learning of these tasks is presumably mediated by the activity of midbrain dopamine (DA) neurons (8). Although DA recordings made while animals are engaged in making such difficult decisions are scarce, experiments on Pavlovian and instrumental conditioning have shown that under a novel stimulus-reward association, DA neurons respond to the unexpected reward with an activity burst. Remarkably, after training this phasic response is shifted to the conditioned stimulus where it works as a signal predicting the future reward (8-12). From a computational standpoint, reinforcement learning (RL) methods (13) have been successfully applied to explain this and many other observations (ref. 14 and for reviews see refs. 15-17). According to the reward prediction error (RPE) hypothesis (18,19), the DA phasic activity signals an error in the prediction of the expected total reward (20)(21)(22) and it is used to learn associations between rewards and task events.In classical and instrumental conditioning the reward acts as a reinforcement, strengthening the association with the stimulus, provided the animal follows the task instructions. In some experiments the reward was delivered only after the animal made a choice between alternative options (20,23,24). However, in those studies the task events were unambiguous: The animals' reports were mostly correct and there was a well-defined temporal relationship between the perceived stimulus and reward delivery. However, this is very different from the real-world situation described above in which the reward is annou...

show abstract

Section: Discussionsupporting

confidence: 53%

“…(B) Each task event was represented across time via a set of functions reproducing the event at different latencies from its onset. Importantly, the resolution of the representation degraded with the passage of time (43).…”

Section: Resultsmentioning

confidence: 99%

Dopamine reward prediction error signal codes the temporal evaluation of a perceptual decision report

Sarno

Lafuente

Parga

2017

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…Operators are commonly used to describe input/output transformation of neuronal signals. Typical examples are those used to model the receptive field of retinal ganglion cells (17), representation of time (18), auditory stimuli (19), and extracellular neural signals (20). In our case, this seems to be a promising approach that allows one to make general predictions about the properties, number, and structure of the operators representing the odor stimuli.…”

Section: Discussionmentioning

confidence: 99%

Synaptic clusters function as odor operators in the olfactory bulb

Migliore

Cavarretta

Marasco

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

How the olfactory bulb organizes and processes odor inputs through fundamental operations of its microcircuits is largely unknown. To gain new insight we focus on odor-activated synaptic clusters related to individual glomeruli, which we call glomerular units. Using a 3D model of mitral and granule cell interactions supported by experimental findings, combined with a matrix-based representation of glomerular operations, we identify the mechanisms for forming one or more glomerular units in response to a given odor, how and to what extent the glomerular units interfere or interact with each other during learning, their computational role within the olfactory bulb microcircuit, and how their actions can be formalized into a theoretical framework in which the olfactory bulb can be considered to contain "odor operators" unique to each individual. The results provide new and specific theoretical and experimentally testable predictions. T he organization of olfactory bulb network elements and their synaptic connectivity has evolved to subserve special computational functions needed for odor detection and recognition (1-5). Key to this organization are the olfactory glomeruli, collecting input from olfactory receptor neuron subsets. These connect to the dendrites of mitral, tufted, and periglomerular cells, and the mitral and tufted cells in turn connect to granule cells. We term these interconnected cells a cluster, and a cluster related to a given glomerulus is a glomerular unit (GU), often visualized as a column of granule cell bodies located below a glomerulus (6, 7). The existence of such GUs has also been suggested from 2-deoxyglucose (8) and voltage-sensitive dye studies (9).Understanding the neural basis of odor processing therefore requires understanding the computational functions and role of GUs. These issues, which are difficult or impossible in experiments, can be conveniently explored using realistic computational models, provided they are able to explain and reproduce crucial experimental findings on glomerular clusters or units.Analyzing synaptic interactions between cells with overlapping dendrites requires modeling in real 3D space. Scaling up to the network level further requires scaling up realistic structural and functional properties to many thousands of cells (10). Building on this unique approach, we show that this model generates columnar clusters of cells related to individual glomeruli, as in the experiments, and further demonstrates mechanisms of odor processing within and between the GUs. Finally, interpreting this network activity requires a theoretical framework, incorporating distributed activated glomeruli within the global network, for which we introduce the concept of the odor operator. The results provide a basis for extension to the glomerular level on the one hand and interactions with olfactory cortex on the other.

show abstract

“…An alternative approach is to assume that different context features change at different temporal rates, directly encoding a representa tion of the time of occurrence of each item. This approach was taken by Howard, Shankar, Aue, and Criss (2015), who showed how such a scale invariant representation of temporal context could simultaneously account for both within-list and across-list recency and contiguity effects (see also Shankar & Howard, 2012).…”

Section: Comparison To Other Models Of Free Recallmentioning

confidence: 99%

Expanding the scope of memory search: Modeling intralist and interlist effects in free recall.

Lohnas¹,

Polyn²,

Kahana³

2015

Psychological Review

180

314

View full text Add to dashboard Cite

The human memory system is remarkable in its capacity to focus its search on items learned in a given context. This capacity can be so precise that many leading models of human memory assume that only those items learned in the context of a recently studied list compete for recall. We sought to extend the explanatory scope of these models to include not only intralist phenomena, such as primacy and recency effects, but also interlist phenomena such as proactive and retroactive interference. Building on retrieved temporal context models of memory search (e.g., Polyn, Norman, & Kahana, 2009), we present a substantially revised theory in which memory accumulates across multiple experimental lists, and temporal context is used both to focus retrieval on a target list, and to censor retrieved information when its match to the current context indicates that it was learned in a nontarget list. We show how the resulting model can simultaneously account for a wide range of intralist and interlist phenomena, including the pattern of prior-list intrusions observed in free recall, build-up of and release from proactive interference, and the ability to selectively target retrieval of items on specific prior lists (Jang & Huber, 2008; Shiffrin, 1970). In a new experiment, we verify that subjects' error monitoring processes are consistent with those predicted by the model.

show abstract

A Scale-Invariant Internal Representation of Time

Cited by 134 publications

References 73 publications

Dopamine reward prediction error signal codes the temporal evaluation of a perceptual decision report

Dopamine reward prediction error signal codes the temporal evaluation of a perceptual decision report

Synaptic clusters function as odor operators in the olfactory bulb

Expanding the scope of memory search: Modeling intralist and interlist effects in free recall.

Contact Info

Product

Resources

About