Tool as Embodiment for Recursive Manipulation

Abstract

Humans and many animals exhibit a robust capability to manipulate diverse objects, often directly with their bodies and sometimes indirectly with tools. Such flexibility is likely enabled by the fundamental consistency in underlying physics of object manipulation such as contacts and force closures. Inspired by viewing tools as extensions of our bodies, we present Tool-As-Embodiment (TAE), a parameterization for tool-based manipulation policies that treat hand-object and tool-object interactions in the same representation space. The result is a single policy that can be applied recursively on robots to use end effectors to manipulate objects, and use objects as tools, i.e. new endeffectors, to manipulate other objects. By sharing experiences across different embodiments for grasping or pushing, our policy exhibits higher performance than if separate policies were trained. Our framework could utilize all experiences from different resolutions of tool-enabled embodiments to a single generic policy for each manipulation skill. Videos at https://sites.google.com/ view/recursivemanipulation

Can Wikipedia Help Offline Reinforcement Learning?

“…Arguably, a fundamental bottleneck for pretraining in RL is the difficulty in reusing a single network across vastly different tasks, of distinct observation spaces, action spaces, rewards, scenes, and agent morphologies. Preliminary work explored various aspects of this problem through graph neural networks for morphology generalization (Wang et al, 2018b;Pathak et al, 2019;Chen et al, 2018;Kurin et al, 2020), language for universal reward specification (Jiang et al, 2019;Lynch & Sermanet, 2021;Shridhar et al, 2022), and object-centric action spaces (Zeng et al, 2020;Shridhar et al, 2022;Noguchi et al, 2021). Our work is orthogonal to these as we essentially amortize RL algorithm itself, expressed as sequence modeling with Transformer, instead of specific RL domain information, and can be combined with domain-specific pre-training techniques (Yen- Chen et al, 2020;Lynch & Sermanet, 2021) effortlessly.…”