Duo: A Human/Wearable Hybrid for Learning About Common Manipulate Objects

Abstract

Abstract. Humanoid robots would benefit from a better understanding of common manipulable objects and the human behaviors associated with them. Duo is a human/wearable hybrid that is designed to learn about this important domain of human intelligence by interacting with natural manipulable objects in unconstrained environments. Duo's wearable Al system measures the kinematic configuration of the human's head, torso and dominant arm, while watching the workspace of the human's hand through a head-mounted camera. Duo also requests helpful actions from the human through speech via headphones. This paper presents results on an initial set of behaviors for Duo which lead to high-quality segmentations of common manipulable objects in unconstrained human environments. In Duo, the wearable AI system essentially subsumes the abilities of its cooperative human partner by sharing the human's sensory input and directing a portion of the human's actions. Together, the cooperative human and the wearable Al system can be thought of as constituting a new kind of humanoid robot that complements more traditional, wholly synthetic humanoid robots by allowing researchers to circumvent some of the currently unsolved problems in the field, from dextrous object manipulation to unrestricted mobility.