An Analysis of Experience Replay in Temporal Difference Learning

Cichosz, Paweł

doi:10.1080/019697299125127

Cited by 22 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Replay has been postulated to solve the temporal credit assignment problem by combining the neuronal representations of both current outcome and prior behavior within a single, brief window of time (Cichosz, 1999;Foster & Wilson, 2006). A natural prediction of this hypothesis is that salience should play an important role in the quantity and/or content of replay.…”

Section: Reward Influences the Content Of Replaymentioning

confidence: 99%

The content of hippocampal “replay”

2018

View full text Add to dashboard Cite

One of the most striking features of the hippocampal network is its ability to self-generate neuronal sequences representing temporally compressed, spatially coherent paths. These brief events, often termed "replay" in the scientific literature, are largely confined to non-exploratory states such as sleep or quiet rest. Early studies examining the content of replay noted a strong correlation between the encoded spatial information and the animal's prior behavior; thus, replay was initially hypothesized to play a role in memory formation and/or systems-level consolidation via "off-line" reactivation of previous experiences. However, recent findings indicate that replay may also serve as a memory retrieval mechanism to guide future behavior or may be an incidental reflection of pre-existing network assemblies. Here, I will review what is known regarding the content of replay events and their correlation with past and future actions, and I will discuss how this knowledge might inform or constrain models which seek to explain the circuit-level mechanisms underlying these events and their role in mnemonic processes.memory, place cells, reactivation, review, ripple | I N TR ODU C TI ONAdaptive behavior requires the brain to preserve coherent representations of experience and later extract that stored information to inform future behaviors. For decades, researchers have utilized goal-directed navigation and the brain's spatial memory system as a specific example to explore more general questions regarding memory processes (Tolman, 1948). Two discoveries in particular have prompted researchers to focus such studies on the hippocampus: (a) Human patients and animal models with damage to their medial temporal lobe (and the hippocampus in particular) display severe impairments in their ability to form new episodic and spatial memories (Morris et al., 1982;Scoville & Milner, 1957;Squire et al., 2004); and (b) individual neurons within the hippocampus represent spatial information in their firing patterns during exploration, implicating the hippocampus in the processing and storage of spatial memories (Moser et al., 2008;O'Keefe and Dostrovsky, 1971). A persistent, fundamental question within this domain is how an entire experience or spatial trajectory, which may be represented within the hippocampus by the activity of tens of thousands of neurons ordered in a precise temporal sequence, can be coherently stored within that neural network or purposefully retrieved at the exact time when the stored information would be useful for guiding future decisions.Among several lines of investigation attempting to address this question (Garner et al., 2012;Josselyn et al., 2015; Ramirez et al., 2013;Tse et al., 2007), hippocampal ripples and ripple-based "replay" have emerged as strong candidate mechanisms which may facilitate both the initial storage and later retrieval of complex, temporally ordered information about experience. Ripples are brief (50-100 ms duration), high-frequency (150-300 Hz) network oscillations within ...

show abstract

Section: Reward Influences the Content Of Replaymentioning

confidence: 99%

The content of hippocampal “replay”

2018

View full text Add to dashboard Cite

show abstract

“…During this process four steps may be distinguished. Modeling can be performed either on the scheme level or the state level [5][6][7][8][9][10]. In the first step two languages are used: JOS -a language defining a narrow subset of natural language to describe the states of the real world, and JWI -a language defining integrity relationships.…”

Section: Modeling Of the Real Worldmentioning

confidence: 99%

Considerations on conceptual knowledge representation

Porwolik

Mulawka

2011

Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2011

View full text Add to dashboard Cite

show abstract

“…As a matter of fact strong artificial intelligence is very close to ontology or even metaphysics. In some retrospective background [1] is summarized about the impact of philosophy on artificial intelligence. In our opinion huge underestimated impact on the concepts of knowledge representation have the following philosophers: Aristotle, Awicenna, Bacon, st. Bonawentura, Descartes, Kant, Leśniewski, Wundt, Chisholm, etc.…”

Section: Introductionmentioning

confidence: 99%

Ontological engineering versus metaphysics

Tataj

Tomanek

Mulawka

2011

Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2011

View full text Add to dashboard Cite

It has been recognized that ontologies are a semantic version of world wide web and can be found in knowledge-based systems. A recent time survey of this field also suggest that practical artificial intelligence systems may be motivated by this research. Especially strong artificial intelligence as well as concept of homo computer can also benefit from their use. The main objective of this contribution is to present and review already created ontologies and identify the main advantages which derive such approach for knowledge management systems. We would like to present what ontological engineering borrows from metaphysics and what a feedback it can provide to natural language processing, simulations and modelling. The potential topics of further development from philosophical point of view is also underlined.

show abstract

An Analysis of Experience Replay in Temporal Difference Learning

Cited by 22 publications

References 0 publications

The content of hippocampal “replay”

The content of hippocampal “replay”

Considerations on conceptual knowledge representation

Ontological engineering versus metaphysics

Contact Info

Product

Resources

About