A Kendama Learning Robot Based on Bi-directional Theory

Miyamoto, Hiroyuki; Schaal, Stefan; Gandolfo, F.; Gomi, Hiroaki; Koike, Yasuharu; Osu, Rieko; Nakano, Eri; Wada, Yasuhiro; Kawato, Mitsuo

doi:10.1016/s0893-6080(96)00043-3

Cited by 141 publications

(76 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In fact, many of the methods that scale to the most interesting tasks are model-based and often employ policy search rather than value function-based approaches , Miyamoto et al, 1996, Bagnell and Schneider, 2001, Kohl and Stone, 2004, Tedrake et al, 2005, Peters and Schaal, 2008b,c, Kober and Peters, 2008. This stands in contrast to perhaps the bulk of [Kaelbling et al, 1996, Sutton andBarto, 1998] research in the machine learning community.…”

Section: (D)mentioning

confidence: 99%

Learning motor skills: from algorithms to robot experiments

Kober

Peters

2014

It - Information Technology

View full text Add to dashboard Cite

Die Veröffentlichung steht unter folgender Creative Commons Lizenz: Namensnennung -Keine kommerzielle Nutzung -Keine Bearbeitung 2.0 Deutschland http://creativecommons.org/licenses/by-nc-nd/2.0/de/ Abstract Ever since the word "robot" was introduced to the English language by KarelČapek's play "Rossum's Universal Robots" in 1921, robots have been expected to become part of our daily lives. In recent years, robots such as autonomous vacuum cleaners, lawn mowers, and window cleaners, as well as a huge number of toys have been made commercially available. However, a lot of additional research is required to turn robots into versatile household helpers and companions. One of the many challenges is that robots are still very specialized and cannot easily adapt to changing environments and requirements. Since the 1960s, scientists attempt to provide robots with more autonomy, adaptability, and intelligence. Research in this field is still very active but has shifted focus from reasoning based methods towards statistical machine learning. Both navigation (i.e., moving in unknown or changing environments) and motor control (i.e., coordinating movements to perform skilled actions) are important sub-tasks.In this thesis, we will discuss approaches that allow robots to learn motor skills. We mainly consider tasks that need to take into account the dynamic behavior of the robot and its environment, where a kinematic movement plan is not sufficient. The presented tasks correspond to sports and games but the presented techniques will also be applicable to more mundane household tasks. Motor skills can often be represented by motor primitives. Such motor primitives encode elemental motions which can be generalized, sequenced, and combined to achieve more complex tasks. For example, a forehand and a backhand could be seen as two different motor primitives of playing table tennis. We show how motor primitives can be employed to learn motor skills on three different levels. First, we discuss how a single motor skill, represented by a motor primitive, can be learned using reinforcement learning. Second, we show how such learned motor primitives can be generalized to new situations. Finally, we present first steps towards using motor primitives in a hierarchical setting and how several motor primitives can be combined to achieve more complex tasks.To date, there have been a number of successful applications of learning motor primitives employing imitation learning. However, many interesting motor learning problems are high-dimensional reinforcement learning problems which are often beyond the reach of current reinforcement learning methods. We review research on reinforcement learning applied to robotics and point out key challenges and important strategies to render reinforcement learning tractable. Based on these insights, we introduce novel learning approaches both for single and generalized motor skills.For learning single motor skills, we study parametrized policy search methods and introduce a framework of reward-weighted imi...

show abstract

Section: (D)mentioning

confidence: 99%

Learning motor skills: from algorithms to robot experiments

Kober

Peters

2014

It - Information Technology

View full text Add to dashboard Cite

show abstract

“…The two subtasks are performed consecutively. Since there is interaction between the first subtask and the second subtask, learning the tennis serve is more difficult than learning kendama (Miyamoto et al, 1996). In order to avoid interaction between subtasks, we carefully selected via-points for the control variable among all of the extracted via-points according to each learning phase.…”

Section: Learning Sequential Movementsmentioning

confidence: 99%

“…One of the essential factors in this theory is the representation of behavior in a sparse via-point representation. Using a Forward Inverse Relaxation Model (FIRM), Kawato (1993, 1995) developed an algorithm to approximately extract the minimum number of via-points, S ¼ {P 1 , P 2 , … , P N }, from a given trajectory, X data , with a level of error threshold d. Generally, (see Kawato et al, 1994 andMiyamoto et al, 1996 for more detail), our approach for learning by watching is as follows. First, a learner estimates the via-points S teacher ¼ X 1 via , X 2 via , … , X N via È É , as the internal representation of the teacher's movement plan by watching the demonstration.…”

Section: Introductionmentioning

confidence: 99%

“…In a previous study, we showed how this approach can be applied to learning the Japanese game of kendama by first extracting via-points from a human demonstration, and then transferring this knowledge to an anthropomorphic robot arm which must perform the same task (Kawato et al, 1994;Miyamoto et al, 1996). We chose the game of kendama as a simple task example.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A tennis serve and upswing learning robot based on bi-directional theory

Miyamoto¹,

Kawato

1998

Neural Networks

Self Cite

View full text Add to dashboard Cite

We experimented on task-level robot learning based on bi-directional theory. The via-point representation was used for 'learning by watching'. In our previous work, we had a robot learn kendama (a Japanese game) in order to demonstrate a single simple task. Our approach can be applied to a wide variety of motor behavior. However, some difficulties still remain. In this paper, we address two problems: (1) how to attain a final goal of complex movement when it consists of a sequence of subgoals, and (2) how to adapt to changes in behavior and the environment. To examine how to solve these problems, we propose two methods: (1) selecting the proper via-points for a control variable for each subgoal, and (2) re-estimating the relation between the via-points and the task during learning without conducting extra trials. We adopted a tennis serve and a pendulum upswing for our complicated tasks. ᭧

show abstract

“…There are several works in robotic imitation to date, each dealing with a specific class of actions; block manipulation (Kuniyoshi and Inoue, 1993;Kuniyoshi et al, 1994), mobile robot navigation (Demiris and Hayes, 1996;Dautenhahn, 1995), dynamic arm motion in a kendama play (Miyamoto et al, 1996), and head motion (Berthouze et al, 1996;Demiris et al, 1997).…”

Section: What Are the Issues In Imitation?mentioning

confidence: 99%

Neural learning of embodied interaction dynamics

Kuniyoshi¹,

Berthouze²

1998

Neural Networks

View full text Add to dashboard Cite

This paper presents our approach towards realizing a robot which can bootstrap itself towards higher complexity through embodied interaction dynamics with the environment including other agents. First, the elements of interaction dynamics are extracted from conceptual analysis of embodied interaction and its emergence, especially of behavioral imitation. Then three case studies are made, presenting our neural architecture and the robotic experiments on some of the important elements discussed above: self exploration and entrainment, emergent coordination, and categorizing self behavior. Finally, we propose that integrating all these elements will be an important step towards realizing the bootstrapping agent envisaged above. ᭧

show abstract

A Kendama Learning Robot Based on Bi-directional Theory

Cited by 141 publications

References 32 publications

Learning motor skills: from algorithms to robot experiments

Learning motor skills: from algorithms to robot experiments

A tennis serve and upswing learning robot based on bi-directional theory

Neural learning of embodied interaction dynamics

Contact Info

Product

Resources

About