Learning compact parameterized skills with a single regression

Stulp, Freek; Raiola, Gennaro; Hoarau, Antoine; Ivaldi, Serena; Sigaud, Olivier

doi:10.1109/humanoids.2013.7030008

Cited by 67 publications

(96 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In all these scenarios, the humans are classically providing the demonstrations (i.e., realizations of the task trajectories) by either manually driving the robot or through teleoperation, following the classical paradigm of imitation learning. Some of them have been also applied to the iCub humanoid robot: for example, Stulp et al (2013) used DMPs to adapt a reaching motion online to the variable obstacles encountered by the robot arm, while Paraschos et al (2015) used ProMPs to learn how to tilt a grate including torque information.…”

Section: Introductionmentioning

confidence: 99%

Prediction of Intention during Interaction with iCub with Probabilistic Movement Primitives

et al. 2017

Self Cite

View full text Add to dashboard Cite

This article describes our open-source software for predicting the intention of a user physically interacting with the humanoid robot iCub. Our goal is to allow the robot to infer the intention of the human partner during collaboration, by predicting the future intended trajectory: this capability is critical to design anticipatory behaviors that are crucial in human-robot collaborative scenarios, such as in co-manipulation, cooperative assembly, or transportation. We propose an approach to endow the iCub with basic capabilities of intention recognition, based on Probabilistic Movement Primitives (ProMPs), a versatile method for representing, generalizing, and reproducing complex motor skills. The robot learns a set of motion primitives from several demonstrations, provided by the human via physical interaction. During training, we model the collaborative scenario using human demonstrations. During the reproduction of the collaborative task, we use the acquired knowledge to recognize the intention of the human partner. Using a few early observations of the state of the robot, we can not only infer the intention of the partner but also complete the movement, even if the user breaks the physical interaction with the robot. We evaluate our approach in simulation and on the real iCub. In simulation, the iCub is driven by the user using the Geomagic Touch haptic device. In the real robot experiment, we directly interact with the iCub by grabbing and manually guiding the robot's arm. We realize two experiments on the real robot: one with simple reaching trajectories, and one inspired by collaborative object sorting. The software implementing our approach is open source and available on the GitHub platform. In addition, we provide tutorials and videos.

show abstract

Section: Introductionmentioning

confidence: 99%

Prediction of Intention during Interaction with iCub with Probabilistic Movement Primitives

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Subsequently, regression methods are used to generalize the optimized contexts to a new, unseen context [Da Silva et al, 2012;Stulp et al, 2013]. Although such approaches have been used successfully, they are time consuming and inefficient in terms of the number of needed training samples as optimizing for different contexts and the generalization between optimized parameters for different contexts are two independent processes.…”

Section: Related Workmentioning

confidence: 99%

Contextual Covariance Matrix Adaptation Evolutionary Strategies

Abdolmaleki

Price

Lau

et al. 2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Many stochastic search algorithms are designed to optimize a fixed objective function to learn a task, i.e., if the objective function changes slightly, for example, due to a change in the situation or context of the task, relearning is required to adapt to the new context. For instance, if we want to learn a kicking movement for a soccer robot, we have to relearn the movement for different ball locations. Such relearning is undesired as it is highly inefficient and many applications require a fast adaptation to a new context/situation. Therefore, we investigate contextual stochastic search algorithms that can learn multiple, similar tasks simultaneously. Current contextual stochastic search methods are based on policy search algorithms and suffer from premature convergence and the need for parameter tuning. In this paper, we extend the well known CMA-ES algorithm to the contextual setting and illustrate its performance on several contextual tasks. Our new algorithm, called contextual CMA-ES, leverages from contextual learning while it preserves all the features of standard CMA-ES such as stability, avoidance of premature convergence, step size control and a minimal amount of parameter tuning.

show abstract

“…Many existing methods [7,33] employ an ensemble approach by learning a discrete set of control policies and consolidating them into one regression model. Our initial attempt with the ensemble approach showed that, for many dynamic tasks, sometimes a small change in the model parameter requires a drastically different control policy to succeed at the given task.…”

Section: A Learning Universal Policymentioning

confidence: 99%

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Yu¹,

Tan²,

Liu³

et al. 2017

Robotics: Science and Systems XIII

164

148

View full text Add to dashboard Cite

Abstract-We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters µ. The value of µ from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.

show abstract

Learning compact parameterized skills with a single regression

Cited by 67 publications

References 12 publications

Prediction of Intention during Interaction with iCub with Probabilistic Movement Primitives

Prediction of Intention during Interaction with iCub with Probabilistic Movement Primitives

Contextual Covariance Matrix Adaptation Evolutionary Strategies

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Contact Info

Product

Resources

About