Switching between Representations in Reinforcement Learning

Seijen, H. van; Whiteson, Shimon; Kester, Leon

doi:10.1007/978-3-642-11688-9_3

Cited by 7 publications

(8 citation statements)

References 9 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although similar methods have also been applied to transfer learning, most of these learn task-relevant representations for SL [2,5] or RL [18,30,37,40,60,70,75,83]; the latter aim to reduce the state space of the target task, or find good representations for value functions or policies. However, none of these approaches learn domain-relevant representations.…”

Section: Representation Learningmentioning

confidence: 99%

Learning potential functions and their representations for multi-task reinforcement learning

Snel

Whiteson

2013

Auton Agent Multi-Agent Syst

Self Cite

View full text Add to dashboard Buy / Rent full text

Section: Representation Learningmentioning

confidence: 99%

Learning potential functions and their representations for multi-task reinforcement learning

Snel

Whiteson

2013

Auton Agent Multi-Agent Syst

Self Cite

View full text Add to dashboard Buy / Rent full text

“…This problem is a modified version of the scenario discussed in [22], which is a stochastic task. There is a crossroad with two-way road resulting in a four-square grid at the center, a horizontal and a vertical one.…”

Section: Case Studies and Resultsmentioning

confidence: 99%

Context Transfer in Reinforcement Learning Using Action-Value Functions

Mousavi

Araabi

Ahmadabadi

2014

Computational Intelligence and Neuroscience

View full text Add to dashboard Buy / Rent full text

This paper discusses the notion of context transfer in reinforcement learning tasks. Context transfer, as defined in this paper, implies knowledge transfer between source and target tasks that share the same environment dynamics and reward function but have different states or action spaces. In other words, the agents learn the same task while using different sensors and actuators. This requires the existence of an underlying common Markov decision process (MDP) to which all the agents' MDPs can be mapped. This is formulated in terms of the notion of MDP homomorphism. The learning framework is Q-learning. To transfer the knowledge between these tasks, the feature space is used as a translator and is expressed as a partial mapping between the state-action spaces of different tasks. The Q-values learned during the learning process of the source tasks are mapped to the sets of Q-values for the target task. These transferred Q-values are merged together and used to initialize the learning process of the target task. An interval-based approach is used to represent and merge the knowledge of the source tasks. Empirical results show that the transferred initialization can be beneficial to the learning process of the target task.

show abstract

“…7 The goal is similar to that of earlier work by Jonsson and Barto (2002) in which each option implemented a separate instance of McCallum's (1996) U-Tree algorithm designed to synthesize state representations from past histories of observations and actions. The goal is also similar to that of Seijen et al (2007) who studied a method that included special abstraction-switching actions.…”

Section: Selecting Skill-specific Abstractionsmentioning

confidence: 97%

Behavioral Hierarchy: Exploration and Representation

Barto

Konidaris

Vigorito

2013

Computational and Robotic Models of the Hierarchical Organization of Behavior

View full text Add to dashboard Buy / Rent full text

Behavioral modules are units of behavior providing reusable building blocks that can be composed sequentially and hierarchically to generate extensive ranges of behavior. Hierarchies of behavioral modules facilitate learning complex skills and planning at multiple levels of abstraction and enable agents to incrementally improve their competence for facing new challenges that arise over extended periods of time. This chapter focusses on two features of behavioral hierarchy that appear to be less well recognized: its influence on exploratory behavior and the opportunity it affords to reduce the representational challenges of planning and learning in large, complex domains. Four computational examples are described that use methods of hierarchical reinforcement learning to illustrate the influence of behavioral hierarchy on exploration and representation. Beyond illustrating these features, the examples provide support for the central role of behavioral hierarchy in development and learning for both artificial and natural agents.

show abstract

Switching between Representations in Reinforcement Learning

Cited by 7 publications

References 9 publications

Learning potential functions and their representations for multi-task reinforcement learning

Learning potential functions and their representations for multi-task reinforcement learning

Context Transfer in Reinforcement Learning Using Action-Value Functions

Behavioral Hierarchy: Exploration and Representation

Contact Info

Product

Resources

About