Relationship Between the Order for Motor Skill Transfer and Motion Complexity in Reinforcement Learning

Cho, Nam Jun; Lee, S.H.; Suh, Il Hong; Kim, Hong-Seok

doi:10.1109/lra.2018.2889026

Cited by 6 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process of actively selecting the task was considered as a non-stationary bandit problem for which a suitable algorithmic solution exists while intrinsic motivation heuristics were exploited to reward the agent after the selection. Cho et al (2019) defined the complexity of a motor skill based on the temporal and spatial entropy of multiple demonstrations and used the measured complexity to generate an order for learning and transferring motor skills. Their experimental findings provided useful guidelines for skill learning and transfer.…”

Section: Dmps Integration In Complex Frameworkmentioning

confidence: 99%

Dynamic movement primitives in robotics: A tutorial survey

Saveriano,

Abu-Dakka,

Kramberger

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

Biological systems, including human beings, have the innate ability to perform complex tasks in a versatile and agile manner. Researchers in sensorimotor control have aimed to comprehend and formally define this innate characteristic. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally leads to the formulation of the motor primitives’ theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant mathematical formulation of the motor primitives as stable dynamical systems and are well suited to generate motor commands for artificial systems like robots. In the last decades, DMPs have inspired researchers in different robotic fields including imitation and reinforcement learning, optimal control, physical interaction, and human–robot co-working, resulting in a considerable amount of published papers. The goal of this tutorial survey is two-fold. On one side, we present the existing DMP formulations in rigorous mathematical terms and discuss the advantages and limitations of each approach as well as practical implementation details. In the tutorial vein, we also search for existing implementations of presented approaches and release several others. On the other side, we provide a systematic and comprehensive review of existing literature and categorize state-of-the-art work on DMP. The paper concludes with a discussion on the limitations of DMPs and an outline of possible research directions.

show abstract

Section: Dmps Integration In Complex Frameworkmentioning

confidence: 99%

Dynamic movement primitives in robotics: A tutorial survey

Saveriano,

Abu-Dakka,

Kramberger

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…The process of actively selecting the task was considered as a non-stationary bandit problem for which suitable algorithmic solution exist while intrinsic motivation heuristics were exploited to reward the agent after the selection. (Cho et al 2019) defined the complexity of a motor skill based on temporal and spatial entropy of multiple demonstrations and used the measured complexity to generate an order for learning and transferring motor skills. Their experimental findings provided useful guidelines for skill learning and transfer.…”

Section: Skills Transfermentioning

confidence: 99%

Dynamic Movement Primitives in Robotics: A Tutorial Survey

Saveriano¹,

Abu‐Dakka²,

Kramberger³

et al. 2021

Preprint

View full text Add to dashboard Cite

Biological systems, including human beings, have the innate ability to perform complex tasks in versatile and agile manner. Researchers in sensorimotor control have tried to understand and formally define this innate property. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally lead to the formulation of the motor primitives theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant mathematical formulation of the motor primitives as stable dynamical systems, and are well suited to generate motor commands for artificial systems like robots. In the last decades, Dynamic Movement Primitives (DMPs) have inspired researchers in different robotic fields including imitation and reinforcement learning, optimal control, physical interaction, and human-robot co-working, resulting a considerable amount of published papers. The goal of this tutorial survey is two-fold. On one side, we present the existing DMPs formulations in rigorous mathematical terms, and discuss advantages and limitations of each approach as well as practical implementation details. In the tutorial vein, we also search for existing implementations of presented approaches and release several others. On the other side, we provide a systematic and comprehensive review of existing literature and categorize state of the art work on DMP. The paper concludes with a discussion on the limitations of DMPs and an outline of possible research directions.

show abstract

“…These problems may prevent robots from acquiring the complete motor skills. Thus, initial motor skills are learned to classify reaction force/moment signals and generate reaction motion trajectories from human demonstrations, and their parameters are improved and/or generalized during the iterations of the improvement/generalization processes [10]. The motor skills learned from human demonstrations are referred to as initial motor skills.…”

mentioning

confidence: 99%

Learning, Improving, and Generalizing Motor Skills for the Peg-in-Hole Tasks Based on Imitation Learning and Self-Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

We propose a framework based on imitation learning and self-learning to enable robots to learn, improve, and generalize motor skills. The peg-in-hole task is important in manufacturing assembly work. Two motor skills for the peg-in-hole task are targeted: "hole search" and "peg insertion". The robots learn initial motor skills from human demonstrations and then improve and/or generalize them through reinforcement learning (RL). An initial motor skill is represented as a concatenation of the parameters of a hidden Markov model (HMM) and a dynamic movement primitive (DMP) to classify input signals and generate motion trajectories. Reactions are classified as familiar or unfamiliar (i.e., modeled or not modeled), and initial motor skills are improved to solve familiar reactions and generalized to solve unfamiliar reactions. The proposed framework includes processes, algorithms, and reward functions that can be used for various motor skill types. To evaluate our framework, the motor skills were performed using an actual robotic arm and two reward functions for RL. To verify the learning and improving/generalizing processes, we successfully applied our framework to different shapes of pegs and holes. Moreover, the execution time steps and path optimization of RL were evaluated experimentally. of motor skills. However, for acquiring complete motor skills, it has one evident limitation: it does not ensure that robots acquire motor skills that are optimized for their goals (that is, it generally provides near-optimal solutions) [5]. Furthermore, it is not easy for human performers to provide a demonstration dataset that can cover all situations arising during the execution of a motor skill [6]. Nonetheless, these human demonstrations can be used as a solid starting point for robots to acquire motor skills [7]. To obtain optimal motor skills, robots must be able to improve motor skills through self-learning. However, this self-learning is a time-consuming and expensive process in the absence of references. We attempt to obtain these optimal solutions with fewer trials-and-errors by providing near-optimal solutions learned from human demonstrations. In this paper, robots improve motor skills to optimize them-referred to as improvement-and generalize them so that they are widely applicable-known as generalization-through self-learning.The peg-in-hole task has also been addressed through several imitation learning studies [8,9]. However, the peg-in-hole task is not easy to learn with this method alone. The main reason is that it is difficult for human performers to provide a complete demonstration dataset to robots, because it is not feasible to prepare all possible reaction situations. In addition, unintended reaction information may be included in the dataset during the demonstration process. These problems may prevent robots from acquiring the complete motor skills. Thus, initial motor skills are learned to classify reaction force/moment signals and generate reaction motion trajectories from human demonstrations, and thei...

show abstract

Relationship Between the Order for Motor Skill Transfer and Motion Complexity in Reinforcement Learning

Cited by 6 publications

References 15 publications

Dynamic movement primitives in robotics: A tutorial survey

Dynamic movement primitives in robotics: A tutorial survey

Dynamic Movement Primitives in Robotics: A Tutorial Survey

Learning, Improving, and Generalizing Motor Skills for the Peg-in-Hole Tasks Based on Imitation Learning and Self-Learning

Contact Info

Product

Resources

About