Gaussian Processes and Reinforcement Learning for Identification and Control of an Autonomous Blimp

Ko, Jonathan; Klein, Daniel J.; Fox, Dieter; Haehnel, Dirk

doi:10.1109/robot.2007.363075

Cited by 153 publications

(157 citation statements)

References 11 publications

Supporting

Mentioning

157

Contrasting

Order By: Relevance

“…In recent years, GP dynamics models were more often used for learning robot dynamics [9,10,14]. However, they are usually not used for long-term planning and policy learning, but rather for myopic control and trajectory following.…”

Section: Related Workmentioning

confidence: 99%

“…However, they are usually not used for long-term planning and policy learning, but rather for myopic control and trajectory following. Typically, the training data for the GP dynamics models are obtained either by motor babbling [9] or by demonstrations [14]. For the purpose of data-efficient fully autonomous learning, these approaches are not suitable: Motor babbling is data-inefficient and does not guarantee good models along a good trajectory; demonstrations would contradict fully autonomous learning.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

Deisenroth

Rasmussen

Fox

2011

Robotics: Science and Systems VII

Self Cite

199

182

View full text Add to dashboard Cite

Abstract-Over the last years, there has been substantial progress in robust manipulation in unstructured environments. The long-term goal of our work is to get away from precise, but very expensive robotic systems and to develop affordable, potentially imprecise, self-adaptive manipulator systems that can interactively perform tasks such as playing with children. In this paper, we demonstrate how a low-cost off-the-shelf robotic system can learn closed-loop policies for a stacking task in only a handful of trials-from scratch. Our manipulator is inaccurate and provides no pose feedback. For learning a controller in the work space of a Kinect-style depth camera, we use a model-based reinforcement learning technique. Our learning method is data efficient, reduces model bias, and deals with several noise sources in a principled way during long-term planning. We present a way of incorporating state-space constraints into the learning process and analyze the learning gain by exploiting the sequential structure of the stacking task.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

Deisenroth

Rasmussen

Fox

2011

Robotics: Science and Systems VII

Self Cite

199

182

View full text Add to dashboard Cite

show abstract

“…Due to the limited space, we simply refer to some approaches which are closely related to the experiments performed in this paper. Ko et al [6] presented an approach to improve a motion model of a blimp derived from aeronautic principles by using a GP to model the residual. Furthermore, Rottmann et al [7] and Deisenroth et al [8] learned control policies of a completely unknown system in a GP framework.…”

Section: Related Workmentioning

confidence: 99%

Learning Non-stationary System Dynamics Online Using Gaussian Processes

Rottmann

Burgard

2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Gaussian processes are a powerful non-parametric framework for solving various regression problems. In this paper, we address the task of learning a Gaussian process model of non-stationary system dynamics in an online fashion. We propose an extension to previous models that can appropriately handle outdated training samples by decreasing their influence onto the predictive distribution. The resulting model estimates for each sample of the training set an individual noise level and thereby produces a mean shift towards more reliable observations. As a result, our model improves the prediction accuracy in the context of non-stationary function approximation and can furthermore detect outliers based on the resulting noise level. Our approach is easy to implement and is based upon standard Gaussian process techniques. In a real-world application where the task is to learn the system dynamics of a miniature blimp, we demonstrate that our algorithm benefits from individual noise levels and outperforms standard methods.

show abstract

“…The approach that is most closely related to the one described in this paper has recently been presented by Ko et al [11]. They also deal with the problem of learning to control an autonomous blimp and choose a similar set of methods for this task.…”

Section: Related Workmentioning

confidence: 99%

Autonomous blimp control using model-free reinforcement learning in a continuous state and action space

Rottmann

Plagemann

Hilgers

et al. 2007

2007 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-In this paper, we present an approach that applies the reinforcement learning principle to the problem of learning height control policies for aerial blimps. In contrast to previous approaches, our method does not require sophisticated handtuned models, but rather learns the policy online, which makes the system easily adaptable to changing conditions. The blimp we apply our approach to is a small-scale vehicle equipped with an ultrasound sensor that measures its elevation relative to the ground. The major problem in the context of learning control policies lies in the high-dimensional state-action space that needs to be explored in order to identify the values of all state-action pairs. In this paper, we propose a solution to learning continuous control policies based on the Gaussian process model. In practical experiments carried out on a real robot we demonstrate that the system is able to learn a policy online within a few minutes only.

show abstract

Gaussian Processes and Reinforcement Learning for Identification and Control of an Autonomous Blimp

Cited by 153 publications

References 11 publications

Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

Learning Non-stationary System Dynamics Online Using Gaussian Processes

Autonomous blimp control using model-free reinforcement learning in a continuous state and action space

Contact Info

Product

Resources

About