Curiosity driven reinforcement learning for motion planning on humanoids

Abstract

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

A Modular Software Framework for Eye–Hand Coordination in Humanoid Robots

“…This way we are able to send arbitrary motions to our system, while ensuring the safety of our robot. Even with just these static objects, this has been shown to provide an interesting way to learn robot reaching behaviors through reinforcement (Pathak et al, 2013;Frank et al, 2014). The presented system has the same functionality also for arbitrary, non-static objects.…”

Extensive assessment and evaluation methodologies on assistive social robots for modelling human–robot interaction – A review

“…So, limited applications to all body movements (1) Temporal Awareness Model/Timing Model; Olsen and Goodrich [122] (2) Human-Robot Team (HRT) Modelling or Tele-Operated Multiple Robot Model Burke et al [14] (3) Robot Awareness Model Kanda et al [80,82] (4) Model of Integrated Humans' shared Intentions, e.g. Haptic Channel, Motion Planning Model through Play Interactions, and the like Steinfeld et al [158] Bethel et al [8] Bethel and Murphy [5] Mutlu et al [119] Cooney et al [25,26] Frank et al [39] Pateraki et al [126] [118] Ease of Classification (EOC) Formula Scoring method for evaluating the Societal Acceptance towards robot…”

“…Kanda et al [83] (4) Task performance metrics [5,8,14,119] [122, 126,158] (may incorporate the subevaluation methodologies of comparisons of the measurement of body movement interaction in between a humanoid robot and humans with subjective evaluation results [25] [25,26,39,82] [83])…”

“…This is based on visual scenes of affectionate play with a small humanoid robot [26], and also through designing very enjoyable motion-based play interactions with a small humanoid robot [25]. In the same vein in this year 2014, Frank et al emphasized on using curiosity driven reinforcement learning for motion planning on humanoids [39] -task performance metrics are important in terms of primary evaluation basis. Kanda et al performed an experiment to evaluate the developed humanoid robot and analyzed the interaction between robots and humans.…”

A neural network model of curiosity-driven infant categorization

“…Curiosity-based intrinsic motivation systems are increasingly used in the development of autonomous computational systems [5], and a growing body of computational and robotic work explores the reward mechanisms which subserve a range of cognitive functions and behaviors, for example low-level perceptual encoding [6], novelty detection [7], reaching [8] and motion planning [9]. Various computational implementations of intrinsic motivation have been proposed, such as the drive to increase the ability to predict outcomes e.g., [10], a "creative" drive to find systematicity in input [11], [12], or competence-based systems which seek out maximally-or minimally difficult tasks (for an in-depth review, see [13]).…”