Improving Reinforcement Learning by Using Case Based Heuristics

Bianchi, Reinaldo A. C.; Ros, Raquel; Mántaras, Ramon López de

doi:10.1007/978-3-642-02998-1_7

Cited by 30 publications

(18 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sharma et al [17] make use of CBR as a function approximator for RL, and RL as revision algorithm for CBR in a hybrid architecture system; Gabel and Riedmiller [18] also makes use of CBR in the task of approximating a function over high-dimensional, continuous spaces; Juell and Paulson [19] exploit the use of RL to learn similarity metrics in response to feedback from the environment; Auslander et al [20] use CBR to adapt quickly an RL agent to changing conditions of the environment by the use of previously stored policies and Li, Zonghai and Feng [21] propose an algorithm that makes use of knowledge acquired by reinforcement learning to construct and extend a case base. Finally, Bianchi, Ros and López de Mántaras [22] use CBR together with Heuristic Accelerated Reinforcement Learning to improve reinforcement learning by using case based heuristics.…”

Section: Transfer Learningmentioning

confidence: 99%

See 1 more Smart Citation

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Celiberto

Matsuura

Mántaras

et al. 2010

2010 Latin American Robotics Symposium and Intelligent Robotics Meeting

View full text Add to dashboard Cite

Abstract-Reinforcement Learning (RL) is a well known technique for the solution of problems where agents need to act with success in an unknown environment, learning through trial and error. However, this technique is not efficient enough to be used in applications with real world demands due to the time that the agent needs to learn. This paper investigates the use of Transfer Learning (TL) between agents to speed up the well known Q-learning Reinforcement Learning algorithm. The new approach presented here allows the use of cases in a case base as heuristics to speed up the Q-learning algorithm, combining Case-Based Reasoning (CBR) and Heuristically Accelerated Reinforcement Learning (HARL) techniques.A set of empirical evaluations were conducted in the Mountain Car Problem Domain, where the actions learned during the solution of the 2D version of the problem can be used to speed up the learning of the policies for its 3D version.The experiments were made comparing the Q-learning Reinforcement Learning algorithm, the HAQL Heuristic Accelerated Reinforcement Learning (HARL) algorithm and the TL-HAQL algorithm, proposed here. The results show that the use of a case-base for transfer learning can lead to a significant improvement in the performance of the agent, making it learn faster than using either RL or HARL methods alone.

show abstract

Section: Transfer Learningmentioning

confidence: 99%

“…To transfer the cases between two learning agents we propose the TL-HAQL (Transfer Learning Heuristically Accelerated Q-learning) algorithm, based in the CB-HAQL algorithm [22].…”

Section: Transfer Learningmentioning

confidence: 99%

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Celiberto

Matsuura

Mántaras

et al. 2010

2010 Latin American Robotics Symposium and Intelligent Robotics Meeting

View full text Add to dashboard Cite

show abstract

“…In contrast, in our work we are using CBR principles to address a wellknown limitation of reinforcement learning. Bianchi et al uses cases as a heuristic to speedup the RL process [7] and Gabel and Riedmiller uses cases to approximate state value functions in continuous spaces [6,17].…”

Section: Related Workmentioning

confidence: 99%

“…For the most part the integration has been aimed at exploiting synergies between RL and CBR that result in performance that is better than each individually (e.g., [3]) or to enhance the performance of the CBR system (e.g., [4]). Although researchers have pointed out that CBR could help to enhance RL processes [5], comparatively little research has been done in this direction, and the bulk of it has concentrated on tasks with continuous states [6,7,16,17].…”

Section: Introductionmentioning

confidence: 99%

Reducing the Memory Footprint of Temporal Difference Learning over Finitely Many States by Using Case-Based Generalization

Dilts

Muñoz-Ávila

2010

Case-Based Reasoning. Research and Development

View full text Add to dashboard Cite

Abstract. In this paper we present an approach for reducing the memory footprint requirement of temporal difference methods in which the set of states is finite. We use case-based generalization to group the states visited during the reinforcement learning process. We follow a lazy learning approach; cases are grouped in the order in which they are visited. Any new state visited is assigned to an existing entry in the Q-table provided that a similar state has been visited before. Otherwise a new entry is added to the Q-table. We performed experiments on a turn-based game where actions have non-deterministic effects and might have long term repercussions on the outcome of the game. The main conclusion from our experiments is that by using case-based generalization, the size of the Q-table can be substantially reduced while maintaining the quality of the RL estimates.

show abstract

“…Case Based Reasoning (CBR) is a knowledge based problem solving technique, which is based on reusing on the previous experiences and has been originated from the researches of cognitive sciences [1]. In this method, it is assumed that the similar problems can possess similar solutions.…”

Section: Introductionmentioning

confidence: 99%

Accelerated Method Based on Reinforcement Learning and Case Base Reasoning in Multi agent Systems

Esfandiari¹,

Masoumi²,

Meybodi³

et al. 2012

IJCA

View full text Add to dashboard Cite

In this paper, a new algorithm based on case base reasoning and reinforcement learning is proposed to increase the rate convergence of the reinforcement learning algorithms in multiagent systems. In the propose method, we investigate how making improved action selection in reinforcement learning (RL) algorithm. In the proposed method, the new combined model using case base reasoning systems and a new optimized function has been proposed to select the action, which has led to an increase in algorithms based on Q-learning. The algorithm mentioned has been used for solving the problem of cooperative Markov's games as one of the models of Markov based multiagent systems. The results of experiments have shown that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy. General TermsMulti Agent Learning , Machine Learning .

show abstract

Improving Reinforcement Learning by Using Case Based Heuristics

Cited by 30 publications

References 16 publications

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Reducing the Memory Footprint of Temporal Difference Learning over Finitely Many States by Using Case-Based Generalization

Accelerated Method Based on Reinforcement Learning and Case Base Reasoning in Multi agent Systems

Contact Info

Product

Resources

About