Learning Bayes Filter Models for Tactile Localization

Abstract

Localizing and tracking the pose of robotic grippers are necessary skills for manipulation tasks. However, the manipulators with imprecise kinematic models (e.g. low-cost arms) or manipulators with unknown world coordinates (e.g. poor camera-arm calibration) cannot locate the gripper with respect to the world. In these circumstances, we can leverage tactile feedback between the gripper and the environment. In this paper, we present learnable Bayes filter models that can localize robotic grippers using tactile feedback. We propose a novel observation model that conditions the tactile feedback on visual maps of the environment along with a motion model to recursively estimate the gripper's location. Our models are trained in simulation with self-supervision and transferred to the real world. Our method is evaluated on a tabletop localization task in which the gripper interacts with objects. We report results in simulation and on a real robot, generalizing over different sizes, shapes, and configurations of the objects.

Learning Contact-Based State Estimation for Assembly Tasks

“…Their policy's objective is to solve a specific manipulation task rather than decrease state uncertainty. Others combine vision and tactile feedback to localize the robot's gripper with respect to the environment, the reverse problem to object pose localization [15].…”

Tactile Pose Estimation and Policy Learning for Unknown Object Manipulation

“…Chaplot et al [8] and Gottipati et al [18] applied this approach to the active localization problem with visual feedback. Finally, in [32], the authors proposed to use a learnable Bayes filter to localize a robotic gripper's position with respect to the environment image using tactile feedback. In the following work [33], they train agents using deep reinforcement learning with belief inputs to solve contact-rich manipulation tasks using only a single image of the environment and the tactile observations.…”