Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization

Turchetta, Matteo; Krause, Andreas; Trimpe, Sebastian

doi:10.1109/icra40945.2020.9197000

Cited by 38 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reference [3] utilizes Gaussian process tool to build a model-based RL path planner for UAS. This can be extended to model-free settings using Reference [71] or other works on similar lines. Extending Reference [70] to policy improvement techniques like actor-critic [36] and policy gradient [69] is also a good direction of future work.…”

Section: Remarksmentioning

confidence: 99%

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Padakandla

2021

ACM Comput. Surv.

View full text Add to dashboard Cite

Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing, and robotics. The real-world complications arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. RL agents in these applications often need to react and adapt to changing operating conditions. A significant part of research on single-agent RL techniques focuses on developing algorithms when the underlying assumption of stationary environment model is relaxed. This article provides a survey of RL methods developed for handling dynamically varying environment models. The goal of methods not limited by the stationarity assumption is to help autonomous agents adapt to varying operating conditions. This is possible either by minimizing the rewards lost during learning by RL agent or by finding a suitable policy for the RL agent that leads to efficient operation of the underlying system. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits. Additionally, we also review works that are tailored to application domains. Finally, we discuss future enhancements for this field.

show abstract

Section: Remarksmentioning

confidence: 99%

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Padakandla

2021

ACM Comput. Surv.

View full text Add to dashboard Cite

show abstract

“…For instance, [14] learns the parameters of a discrete event controller for a bipedal robot with BO, while [15], trades off real-world and simulated control experiments via BO. In [16], variational autoencoders are combined with BO to learn to control an hexapod, while [17] uses multiobjective BO to learn robust controllers for a pendulum.…”

Section: Related Workmentioning

confidence: 99%

Safe and Efficient Model-free Adaptive Control via Bayesian Optimization

König

Turchetta

Lygeros

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Adaptive control approaches yield highperformance controllers when a precise system model or suitable parametrizations of the controller are available. Existing data-driven approaches for adaptive control mostly augment standard model-based methods with additional information about uncertainties in the dynamics or about disturbances. In this work, we propose a purely data-driven, model-free approach for adaptive control. Tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance. Thus, our approach builds on GOOSE, an algorithm for safe and sample-efficient Bayesian optimization. We introduce several computational and algorithmic modifications in GOOSE that enable its practical use on a rotational motion system. We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation. We further demonstrate the proposed adaptive control approach experimentally on a rotational motion system.

show abstract

“…[54] utilizes Gaussian process tool to build a model-based RL path planner for UAS. This can be extended to model-free settings using [69] or other works on similar lines. Extending [61] to policy improvement techniques like actor-critic [70] and policy gradient [71] is also a good direction of future work.…”

Section: F Remarksmentioning

confidence: 99%

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Padakandla

2020

Preprint

View full text Add to dashboard Cite

Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing and robotics. The realworld complications of many tasks arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. RL agents in these applications often need to react and adapt to changing operating conditions. A significant part of research on single-agent RL techniques focuses on developing algorithms when the underlying assumption of stationary environment model is relaxed. This paper provides a survey of RL methods developed for handling dynamically varying environment models. The goal of methods not limited by the stationarity assumption is to help autonomous agents adapt to varying operating conditions. This is possible either by minimizing the rewards lost during learning by RL agent or by finding a suitable policy for the RL agent which leads to efficient operation of the underlying system. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits. Additionally we also review works which are tailored to application domains. Finally, we discuss future enhancements for this field.

show abstract

Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization

Cited by 38 publications

References 18 publications

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Safe and Efficient Model-free Adaptive Control via Bayesian Optimization

A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Contact Info

Product

Resources

About