Morphology-Agnostic Visual Robotic Control

Abstract

Existing approaches for visuomotor robotic control typically require characterizing the robot in advance by calibrating the camera or performing system identification. We propose MAVRIC, an approach that works with minimal prior knowledge of the robot's morphology, and requires only a camera view containing the robot and its environment and an unknown control interface. MAVRIC revolves around a mutual informationbased method for self-recognition, which discovers visual "control points" on the robot body within a few seconds of exploratory interaction, and these control points in turn are then used for visual servoing. MAVRIC can control robots with imprecise actuation, no proprioceptive feedback, unknown morphologies including novel tools, unknown camera poses, and even unsteady handheld cameras. We demonstrate our method on visuallyguided 3D point reaching, trajectory following, and robot-torobot imitation.

Know Thyself: Transferable Visual Control Policies Through Robot-Awareness

“…Robotic self-recognition literature assumes limited or no self-knowledge, and aims to identify the robot in visual observations [14,17,9]. Recently, Yang et al [29] propose an approach for fast, coarse-grained visual self-recognition and control of the robot's end-effector. We assume knowledge of robot morphology, dynamics, and camera calibration, which are commonly available or easy to acquire in many settings.…”