Learning and generalization of motor skills by learning from demonstration

Abstract-Dynamic Movement Primitives (DMPs) are widely used for encoding motion data. Task parameterized DMP (TP-DMP) can adapt a learned skill to different situations. Mostly a customized vision system is used to extract task specific variables. This limits the use of such systems to real world scenarios. This paper proposes a method for combining the DMP with a Convolutional Neural Network (CNN). Our approach preserves the generalization properties associated with a DMP, while the CNN learns the task specific features from the camera images. This eliminates the need to extract the task parameters, by directly utilizing the camera image during the motion reproduction. The performance of the developed approach is demonstrated through a trash cleaning task, executed with a real robot. We also show that by using the data augmentation, the learned sweeping skill can be generalized for arbitrary objects. The experiments show the robustness of our approach for several different settings.

show abstract

“…In this work we consider the DMP formulation presented in [10]. Each degree of freedom (DoF) is encoded by a separate DMP.…”

Section: Movement Primitives a Dynamic Movement Primitivementioning

confidence: 99%

Learning deep movement primitives using convolutional neural networks

Pervez

Mao

Lee

2017

2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)

View full text Add to dashboard Cite

show abstract

“…al. [9] used a Dynamic Movement Primitive (DMP) framework in which the recorded movement were represented using nonlinear differential equations. The movement library consisted of actions such as grasping, placing and releasing.…”

Section: Related Workmentioning

confidence: 99%

Language for learning complex human-object interactions

Patel

Kyriazis

et al. 2013

2013 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

Abstract-In this paper we use a Hierarchical Hidden Markov Model (HHMM) to represent and learn complex activities/task performed by humans/robots in everyday life. Action primitives are used as a grammar to represent complex human behaviour and learn the interactions and behaviour of human/robots with different objects. The main contribution is the use of a probabilistic model capable of representing behaviours at multiple levels of abstraction to support the proposed hypothesis. The hierarchical nature of the model allows decomposition of the complex task into simple action primitives. The framework is evaluated with data collected for tasks of everyday importance performed by a human user.

show abstract

“…Imitation learning has been addressed frequently in robotics for enabling bootstrapping in object grasping and interaction with the environment in general, [28], [8], [20], [27], [2]. An important challenge in imitation learning is the "correspondence problem" [21] due to the differences in embodiments between humans and robots, being especially problematic in grasping applications [32], [30], see an example in Fig.…”

Section: Introductionmentioning

confidence: 99%

Predicting human intention in visual observations of hand/object interactions

Song

Kyriazis

Oikonomidis

et al. 2013

2013 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

Abstract-The main contribution of this paper is a probabilistic method for predicting human manipulation intention from image sequences of human-object interaction. Predicting intention amounts to inferring the imminent manipulation task when human hand is observed to have stably grasped the object. Inference is performed by means of a probabilistic graphical model that encodes object grasping tasks over the 3D state of the observed scene. The 3D state is extracted from RGB-D image sequences by a novel vision-based, markerless hand-object 3D tracking framework. To deal with the high-dimensional statespace and mixed data types (discrete and continuous) involved in grasping tasks, we introduce a generative vector quantization method using mixture models and self-organizing maps. This yields a compact model for encoding of grasping actions, able of handling uncertain and partial sensory data. Experimentation showed that the model trained on simulated data can provide a potent basis for accurate goal-inference with partial and noisy observations of actual real-world demonstrations. We also show a grasp selection process, guided by the inferred human intention, to illustrate the use of the system for goal-directed grasp imitation.

show abstract

Learning and generalization of motor skills by learning from demonstration

Cited by 565 publications

References 9 publications

Learning deep movement primitives using convolutional neural networks

Learning deep movement primitives using convolutional neural networks

Language for learning complex human-object interactions

Predicting human intention in visual observations of hand/object interactions

Contact Info

Product

Resources

About