Stable reinforcement learning with autoencoders for tactile and visual data

Abstract

For many tasks, tactile or visual feedback is helpful or even crucial. However, designing controllers that take such high-dimensional feedback into account is non-trivial. Therefore, robots should be able to learn tactile skills through trial and error by using reinforcement learning algorithms. The input domain for such tasks, however, might include strongly correlated or non-relevant dimensions, making it hard to specify a suitable metric on such domains. Auto-encoders specialize in finding compact representations, where defining such a metric is likely to be easier. Therefore, we propose a reinforcement learning algorithm that can learn non-linear policies in continuous state spaces, which leverages representations learned using auto-encoders. We first evaluate this method on a simulated toytask with visual input. Then, we validate our approach on a real-robot tactile stabilization task.

Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks

“…Many of them are concerned with estimating the stability of a grasp before lifting an object [6,14], even suggesting a regrasp [60]. Only a few approaches learn entire manipulation policies through reinforcement only given haptic feedback [29,30,[61][62][63]65]. While [30] relies on raw force-torque feedback, [29,61,62] learn a low-dimensional representation of high-dimensional tactile data before learning a policy, and [63] learns a dynamics model of the tactile feedback in a latent space.…”

“…A popular representation learning objective is reconstruction of the raw sensory input through variational autoencoders [11,29,40,70], which we consider as a baseline in this work. This unsupervised objective benefits learning stability and speed, but it is also data intensive and prone to overfitting [11].…”

Affordance Learning for End-to-End Visuomotor Robot Control

“…One approach to solve this issue is to introduce extra loss functions to extract general multitask features. A popular solution is to learn lowdimensional state representations of the visual data [1], [11], [16], [17]. Encoder-decoder architectures (also known as auto-encoders when the input data is restored at the output) are widely used models for this purpose.…”

“…Overview of different priors for the perception training problem can be found in [21] and [22]. We train a lowdimensional representation of visual inputs using variational encoder-decoder structures, which, similar to [16], [17], assigns a normal distribution over the latent space conditioned on the observations. This prevents similar states from being scattered in the latent space, which is a suitable property for control purposes.…”

Automatic Grasping Using Tactile Sensing and Deep Calibration

“…A combination of both of them through deep architecture is a promising solution in [8], [9]. However, processing these high-dimensional data is not an easy task and a meaningful compact representation would be needed [10], [11]. A robot can learn the manipulation using tactile sensation through demonstrations [12], [10], [13].…”