Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks

Rozo, Leonel; Guo, Meng; Kupcsik, Andras; Todescato, Marco; Schillinger, Philipp; Giftthaler, Markus; Ochs, Matthias; Spies, Markus; Waniek, Nicolai; Kesper, Patrick; Bürger, Mathias

doi:10.1109/iros45743.2020.9341570

Cited by 21 publications

(37 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Manipulation skills on R 3 × S 3 : To test our approach in a robotic setting, we consider a re-orient skill from [27], which involves lifting a previously-grasped object, rotating the endeffector, and placing the object back on its original location with a modified orientation (see Fig. 5).…”

Section: Methodsmentioning

confidence: 99%

“…where µ + is mean of the resulting Gaussian (that we are estimating) and Λ y,s = Γ µy,s→µ + (Σ −1 y,s ). We can rewrite (27) by defining the vector (µ + ) = Log µ + (µ y,1 ) T • • • Log µ + (µ y,S ) T T and the block diagonal matrix Λ = blockdiag(Λ y,1 , • • • , Λ y,S ). This results in J having the form of the objective function used to compute the empirical mean v of a Gaussian distribution on a Riemannian manifold M (Eq.…”

Section: B3 Blendingmentioning

confidence: 99%

See 1 more Smart Citation

Orientation Probabilistic Movement Primitives on Riemannian Manifolds

Rozo¹,

Dave²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Learning complex robot motions necessarily demands to have models that are able to encode and retrieve full-pose trajectories when tasks are defined in operational spaces. Probabilistic movement primitives (ProMPs) stand out as a principled approach that models trajectory distributions learned from demonstrations. ProMPs allow for trajectory modulation and blending to achieve better generalization to novel situations. However, when ProMPs are employed in operational space, their original formulation does not directly apply to full-pose movements including rotational trajectories described by quaternions. This paper proposes a Riemannian formulation of ProMPs that enables encoding and retrieving of quaternion trajectories. Our method builds on Riemannian manifold theory, and exploits multilinear geodesic regression for estimating the ProMPs parameters. This novel approach makes ProMPs a suitable model for learning complex full-pose robot motion patterns. Riemannian ProMPs are tested on toy examples to illustrate their workflow, and on real learning-from-demonstration experiments.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: B3 Blendingmentioning

confidence: 99%

Orientation Probabilistic Movement Primitives on Riemannian Manifolds

Rozo¹,

Dave²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hidden Markov models (HMMs) have been typically used as a generative model for recognition and generation of movement skills in robotics (Asfour et al, 2008; Duque et al, 2019; Lee and Ott, 2010; Mor et al, 2021; Rozo et al, 2020; Tanwani and Calinon, 2017; Vakanski et al, 2012). For example, Kulic et al (2008) used HMMs to incrementally group whole-body motions based on their relative distance in HMM space.…”

Section: Related Workmentioning

confidence: 99%

Sequential robot imitation learning from observations

Tanwani

Yan

Calinon

et al. 2021

The International Journal of Robotics Research

View full text Add to dashboard Cite

This paper presents a framework to learn the sequential structure in the demonstrations for robot imitation learning. We first present a family of task-parameterized hidden semi-Markov models that extracts invariant segments (also called sub-goals or options) from demonstrated trajectories, and optimally follows the sampled sequence of states from the model with a linear quadratic tracking controller. We then extend the concept to learning invariant segments from visual observations that are sequenced together for robot imitation. We present Motion2Vec that learns a deep embedding space by minimizing a metric learning loss in a Siamese network: images from the same action segment are pulled together while being pushed away from randomly sampled images of other segments, and a time contrastive loss is used to preserve the temporal ordering of the images. The trained embeddings are segmented with a recurrent neural network, and subsequently used for decoding the end-effector pose of the robot. We first show its application to a pick-and-place task with the Baxter robot while avoiding a moving obstacle from four kinesthetic demonstrations only, followed by suturing task imitation from publicly available suturing videos of the JIGSAWS dataset with state-of-the-art [Formula: see text]% segmentation accuracy and [Formula: see text] cm error in position per observation on the test set.

show abstract

“…Unit quaternions are a convenient way to represent an orientation since they are compact, not redundant, and prevent gimbal lock. Also, they provide strong stability guarantees in closed-loop orientation control [7], they have been recently exploited in robot skills learning [34], and for data-efficient robot control tuning [21] under a Riemannianmanifold formulation.…”

Section: B Orientation Encoding On Smentioning

confidence: 99%

“…The first set of experiments is based on a dataset collected while a human operator performs kinesthetic demonstrations of a grasping skill. This particular grasping motion includes a 90°rotation when approaching the object for performing a side grasp [34]. The demonstration trajectories start from different end-effector poses, and they reach the same target position with slightly different orientations.…”

Section: Real Grasping Taskmentioning

confidence: 99%

Learning Riemannian Manifolds for Geodesic Motion Skills

Beik-Mohammadi

Hauberg²,

Arvanitidis

et al. 2021

Robotics: Science and Systems XVII

Self Cite

View full text Add to dashboard Cite

For robots to work alongside humans and perform in unstructured environments, they must learn new motion skills and adapt them to unseen situations on the fly. This demands learning models that capture relevant motion patterns, while offering enough flexibility to adapt the encoded skills to new requirements, such as dynamic obstacle avoidance. We introduce a Riemannian manifold perspective on this problem, and propose to learn a Riemannian manifold from human demonstrations on which geodesics are natural motion skills. We realize this with a variational autoencoder (VAE) over the space of position and orientations of the robot end-effector. Geodesic motion skills let a robot plan movements from and to arbitrary points on the data manifold. They also provide a straightforward method to avoid obstacles by redefining the ambient metric in an online fashion. Moreover, geodesics naturally exploit the manifold resulting from multiple-mode tasks to design motions that were not explicitly demonstrated previously. We test our learning framework using a 7-DoF robotic manipulator, where the robot satisfactorily learns and reproduces realistic skills featuring elaborated motion patterns, avoids previously-unseen obstacles, and generates novel movements in multiple-mode settings.

show abstract

Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks

Cited by 21 publications

References 26 publications

Orientation Probabilistic Movement Primitives on Riemannian Manifolds

Orientation Probabilistic Movement Primitives on Riemannian Manifolds

Sequential robot imitation learning from observations

Learning Riemannian Manifolds for Geodesic Motion Skills

Contact Info

Product

Resources

About