Towards Rapid Multi-robot Learning from Demonstration at the RoboCup Competition

Abstract

Abstract. We describe our previous and current efforts towards achieving an unusual personal RoboCup goal: to train a full team of robots directly through demonstration, on the field of play at the RoboCup venue, how to collaboratively play soccer, and then use this trained team in the competition itself. Using our method, HiTAB, we can train teams of collaborative agents via demonstration to perform nontrivial joint behaviors in the form of hierarchical finite-state automata. We discuss HiTAB, our previous efforts in using it in RoboCup 2011 and 2012, recent experimental work, and our current efforts for 2014, then suggest a new RoboCup Technical Challenge problem in learning from demonstration.Imagine that you are at an unfamiliar disaster site with a team of robots, and are faced with a previously unseen task for them to do. The robots have only rudimentary but useful utility behaviors implemented. You are not a programmer. Without coding them, you have only a few hours to get your robots doing useful collaborative work in this new environment. How would you do this?Our interest lies in rapid, real-time multi-robot training from demonstration. Here a single human trainer teaches a team of robots, via teleoperation, how to collectively perform tasks in previously unforeseen environments. This is difficult for two reasons. First, nontrivial behaviors can present a high-dimensional space to learn, yet one can only provide a few samples, as online training samples are costly to collect. This is a worst case for the so-called "curse of dimensionality". Second, when training multiple interactive robots, even if you can quantify the emergent macro-level group behavior you wish to achieve, in order to do learning, each agent needs to know the micro-level behavior he is being asked to do. One may have a micro→macro function (a simulator), but it is unlikely that one has the inverse macro→micro function, resulting in what we call the "multiagent inverse problem". These two challenges mean that real-time multi-robot learning from demonstration has proven very difficult and has a very sparse literature.Over the past several years we have participated in the Kid-Size Humanoid League with a single objective: to successfully do a personal RoboCup-style technical challenge of our own invention, independent of those offered at RoboCup: can we train multiple generic robots, through demonstration on the field, how to play collaborative soccer at RoboCup solely within the preparatory time prior to the competition itself ?

A Dataset Schema for Cooperative Learning from Demonstration in Multi-robot Systems

“…A promising approach based on LfD consists of hierarchical task decomposition [11]. The goal is to split complex behaviors into simpler tasks.…”

“…Many works have developed solutions to create setplays in the robotic soccer domain [18][10] [1] [11][4] [27]. Section 2 presents details about these solutions and tools.…”

“…The results of the proposed solution are validated in a Multi-Robot System (MRS) organizing a collection of new cooperative plans recommendations from the demonstration by domain experts. Keywords Multi-Agent System · Learning from Demonstration · Dataset · Coordination · Multi-robot plan · clustering PACS 07.05.Mh Dataset Schema for Cooperative Learning in a MRS 3Learning from Demonstration (LfD) has been used to learn robots basic skills or even very simple setplays in a MRS [11]. Although, there is no register of using LfD to learn complex setplays.…”

DART: Diversity-enhanced Autonomy in Robot Teams

“…However, learning from human cooperation requires multi-agent learning tools for many agents. Although there is recent interest in learning from demonstration for MRS (Chernova and Veloso, 2010; Freelan et al, 2015; Martins and Demiris, 2010), these works either require a significant amount of domain knowledge, leading to potential bias in the creation of design policies, or do not apply to tightly coordinated tasks. New approaches to learning are necessary in order to learn truly novel behaviors from demonstration.…”

Using Neural Networks for Programming by Demonstration

“…These models depend on prior specification of possible behavior. Hierarchical training methods [17], [18] require that the demonstrator manually decompose the task into a hierarchy of subtasks. Our work does not impose such a requirement on the demonstrator.…”