Achieving versatile manipulation tasks with unknown objects by supervised humanoid robots based on object templates

Abstract

The investigations of this paper are motivated by the scenario of a supervised semi-autonomous humanoid robot entering a mainly unknown, potentially degraded human environment to perform highly diverse disaster recovery tasks. For this purpose, the robot must be enabled to use any object it can find in the environment as tool for achieving its current manipulation task. This requires the use of potential unknown objects as well as known objects for new purposes (e.g. using a drill as a hammer). A recently proposed object template manipulation approach is extended to provide a semi-autonomous humanoid robot assisted by a remote human supervisor with the versatility needed to utilize objects in the described manner applying affordances [1] from other previously known objects. For an Atlas humanoid robot it is demonstrated how using a small set of such object templates with well defined affordances can be used to solve manipulation tasks using new unknown objects.

Collaborative Autonomy between High‐level Behaviors and Human Operators for Remote Manipulation Tasks using Different Humanoid Robots

“…During robot experimentation, it was discovered that the object template approach allows flexibility to adapt to situations in which an object in the environment is found but no object template has been created for that object yet. This flexibility extends the use of the object template approach to achieve manipulation tasks with unknown objects, with intermediary objects, or to use objects in a way they were not designed for (Romay, Kohlbrecher, Conner, & von Stryk, ).…”

Towards More Robotic Assistance for Everyday Life

“…In particular, manipulating unknown objects and interacting with tools in complex, unstructured environments is difficult and requires understanding an object's function. Template manipulation skills and affordances of objects are promising approaches to address these challenges (Romay et al, 2015). In Starke et al, a new deep neural autoencoder for human grasps or unknown objects has been developed to better transfer these skills to robots.…”

Augmented Reality Views for Occluded Interaction

“…To structure the space of occluded interaction, we looked into task taxonomies [4,7,18,26,38]. From this we built a selection of tasks that is representative of the range of movement complexities and constraints [7].…”