Programming service tasks in household environments by human demonstration

Imitation Learning of Whole-Body Grasps

“…In the 'what is being learned' area, most imitation learning research focuses on learning assembly/pick-and-place operations (Brand, 1997), (Ehrenmann, 2002), (Kang, 1991), (Kuniyoshi, 1994), (Ogata, 1994), (Paul, 1996), (Tung, 1995). Such research generally looks at an input data stream, and attempts to segment the stream to identify actions performed by a human hand.…”

“…These include hidden Markov models/finite state automata (Brand, 1997), (Ogata, 1994), (Paul, 1996); relational/contact expression grammars (Kuniyoshi, 1994), (Pangburn, 1994), (Siskind, 2000), (Tung, 1995), in which properties such as 'is-picking-up', 'is-touching' and 'infront-of' are used to reason about the state of assembly tasks; contact wrenches (Pollard, 2002), in which a task is represented by a sequence of torques at contact points; contact locations (Kang, 1993), (Kang, 1991) or locations and forces together (Ehrenmann, 2002) of a hand on an object, which are used to determine which grasp in a taxonomy is being used; perceptuo-motor primitives consisting of lines and arcs of motion in the air (Jenkins, 2000) or air hockey movements (Bentivegna, 2002); linear quadratic regulators (LQR) for pole-balancing (Schall, 1997); or an assortment of properties related to specific verbs like push or shove, such as elbow extension or acceleration (Bailey, 1998).…”

“…Finally, in the 'how data is input' area, all of the above work uses videos or motion capture of humans performing tasks (Bentivegna, 2002), (Brand, 1997), (Jenkins, 2000), (Kang, 1993), (Kuniyoshi, 1994), (Pangburn, 1994), (Paul, 1996), (Pollard, 2002), (Schaal, 1997), (Siskind, 2000) or teleoperation using sensors hooked to a simulated world (Bailey, 1998), (Bentivegna, 2002), (Ehrenmann, 2002), (Ogata, 1994), (Tung, 1995).…”

Leveraging on a virtual environment for robot programming by demonstration

“…Recent examples of PbD systems involving demonstration in the real world are [5,8,9]. This is also the most general approach to programming by demonstration, but the complexity of the underlying recognition and interpretation techniques strongly constrains its applicability.…”

“…In service robotics applications the simulation feature is even more important since it has been pointed out that, in many cases, it would be almost impossible to ask for multiple demonstrations by the user [8].…”