Advances and Challenges in Learning from Experience Replay

Abstract

From the first theoretical propositions in the ’50s, inspired by the neu-roscience and psychology studies about the learning processes in humanbeings and animals, to its application in real-world problems on learn-ing to action, Reinforcement Learning (RL) is still being a fascinating,rich, and complex class of machine learning algorithms. In particular,we will start reviewing its fundamental principles and develop a discus-sion about how a technique called Experience Replay (ER) has beenof fundamental importance in making a variety of methods in most ofthe relevant problems and different domains more data efficient, usingagent experiences to improve its performance. We present some of themore relevant methods in the literature, which base most recent researchon improving RL with ER. Finally, we bring from the recent literaturesome of the main trends, challenges, and advances focused on reviewingand discussing ER formal basement and how to improve its propositionsto make it even more efficient in different methods and domains.