Transfer in Reinforcement Learning: A Framework and a Survey

Lazaric, Alessandro

doi:10.1007/978-3-642-27645-3_5

Cited by 213 publications

(156 citation statements)

References 38 publications

Supporting

Mentioning

146

Contrasting

Unclassified

Order By: Relevance

“…In this scenario, usually referred 25 to as multi-task reinforcement learning (MTRL), the 26 objective is to simultaneously solve multiple tasks and 27 exploit their similarity to improve the performance w.r.t. 28 single-task learning (we refer to [28] and [16] for a com- …”

mentioning

confidence: 99%

See 1 more Smart Citation

Sparse multi-task reinforcement learning

Calandriello

Lazaric

Restelli

2015

Self Cite

View full text Add to dashboard Cite

U n c o r r e c t e d A u t h o r P r o o fIntelligenza Artificiale xx (20xx) Abstract. In multi-task reinforcement learning (MTRL), the objective is to simultaneously learn multiple tasks and exploit their similarity to improve the performance w.r.t. single-task learning. In this paper we investigate the case when all the tasks can be accurately represented in a linear approximation space using the same small subset of the original (large) set of features. This is equivalent to assuming that the weight vectors of the task value functions are jointly sparse, i.e., the set of their non-zero components is small and it is shared across tasks. Building on existing results in multi-task regression, we develop two multi-task extensions of the fitted Q-iteration algorithm. While the first algorithm assumes that the tasks are jointly sparse in the given representation, the second one learns a transformation of the features in the attempt of finding a more sparse representation. For both algorithms we provide a sample complexity analysis and numerical simulations.

show abstract

mentioning

confidence: 99%

“…Results of the first experiment in the chain walk domain comparing GL-FQI and LASSO-FQI. On the y axis we have the average regret computed according to Equation (16). On the x axis we have the total number of dimensions d, including noise dimensions, on a logarithmic scale.…”

mentioning

confidence: 99%

Sparse multi-task reinforcement learning

Calandriello

Lazaric

Restelli

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…RL has a strong relationship with supervised learning and unsupervised learning, so it is quite appealing to introduce more machine learning methods to RL problems. For example, there have been some research on combining transfer learning with RL [114] , aiming to solve different tasks with transferred knowledge. When the training data set is large, the computational cost of batch RL algorithms will become a serious problem.…”

Section: Discussionmentioning

confidence: 99%

Feature selection and feature learning for high-dimensional batch reinforcement learning: A survey

Liu

Wang

2015

Int. J. Autom. Comput.

View full text Add to dashboard Cite

Tremendous amount of data are being generated and saved in many complex engineering and social systems every day. It is significant and feasible to utilize the big data to make better decisions by machine learning techniques. In this paper, we focus on batch reinforcement learning (RL) algorithms for discounted Markov decision processes (MDPs) with large discrete or continuous state spaces, aiming to learn the best possible policy given a fixed amount of training data. The batch RL algorithms with handcrafted feature representations work well for low-dimensional MDPs. However, for many real-world RL tasks which often involve high-dimensional state spaces, it is difficult and even infeasible to use feature engineering methods to design features for value function approximation. To cope with high-dimensional RL problems, the desire to obtain data-driven features has led to a lot of works in incorporating feature selection and feature learning into traditional batch RL algorithms. In this paper, we provide a comprehensive survey on automatic feature selection and unsupervised feature learning for high-dimensional batch RL. Moreover, we present recent theoretical developments on applying statistical learning to establish finite-sample error bounds for batch RL algorithms based on weighted Lp norms. Finally, we derive some future directions in the research of RL algorithms, theories and applications.

show abstract

“…5). In addition, it has been shown that some learning mechanisms, such as reinforcement learning and transfer learning, can be helpful for constructing more complex intelligent reasoning systems (Lazaric, 2012). Furthermore, lifelong learning (Lazer et al, 2014) is the key capability of advanced intelligence systems.…”

Section: Cross-media Knowledge Evolution and Reasoningmentioning

confidence: 99%

Cross-media analysis and reasoning: advances and directions

Peng

Zhu

Zhao

et al. 2017

Frontiers Inf Technol Electronic Eng

View full text Add to dashboard Cite

Abstract:Cross-media analysis and reasoning is an active research area in computer science, and a promising direction for artificial intelligence. However, to the best of our knowledge, no existing work has summarized the state-of-the-art methods for cross-media analysis and reasoning or presented advances, challenges, and future directions for the field. To address these issues, we provide an overview as follows: (1) theory and model for cross-media uniform representation; (2) cross-media correlation understanding and deep mining; (3) cross-media knowledge graph construction and learning methodologies; (4) cross-media knowledge evolution and reasoning; (5) cross-media description and generation; (6) cross-media intelligent engines; and (7) cross-media intelligent applications. By presenting approaches, advances, and future directions in cross-media analysis and reasoning, our goal is not only to draw more attention to the state-of-the-art advances in the field, but also to provide technical insights by discussing the challenges and research directions in these areas.

show abstract

Transfer in Reinforcement Learning: A Framework and a Survey

Cited by 213 publications

References 38 publications

Sparse multi-task reinforcement learning

Sparse multi-task reinforcement learning

Feature selection and feature learning for high-dimensional batch reinforcement learning: A survey

Cross-media analysis and reasoning: advances and directions

Contact Info

Product

Resources

About