Human-Scale Mobile Manipulation Using RoMan

Abstract

We present the design, integration, and evaluation of a full-stack robotic system called RoMan, which can conduct autonomous field operations involving physical interaction with its environment. RoMan offers autonomous behaviors that can be triggered from succinct, high-level human input such as “open this box and retrieve the bag inside.” The robot’s behaviors are driven by a set of planners and controllers grounded in perceptual reconstructions of the environment. These behaviors are articulated by a behavior tree that translates high-level operator input into programs of increasing sensorimotor expressiveness, ultimately driving the lowest-level controllers. The software system is implemented in ROS as a set of independent processes connected by synchronous and asynchronous communication, and distributed across two on-board planning/control computers. The behavior stack drives a novel platform consisting of a pair of custom, 500 Nm/axis manipulators mounted on a rotatable torso aboard a tracked platform. The robot’s head is equipped with forward-looking depth cameras, and the arms carry wrist-mounted force-torque sensors and a mix of three- and four-finger grippers. We discuss design and implementation trade-offs affecting the entire hardware-software stack and high-level manipulation behaviors. We also demonstrate the applicability of the system for solving two manipulation tasks: 1) removing heavy debris from a roadway, where 64% of end-to-end autonomous runs required at most one human intervention; and 2) retrieving an item from a closed container, with a fully autonomous success rate of 56%. Finally, we indicate lessons learned and suggest outstanding research problems.

Grasping and Transport of Unstructured Collections of Massive Objects

“…While the grasp planner was employed extensively across a variety of manipulation behaviors documented in (Kessens et al, 2021), a restricted set of experiments was conducted to directly validate its capacity to grasp representative objects for the human-scale urban environment within which the system is intended to operate. Trials were conducted on the aluminium truss segment, 2"x4" wood section, and safety barrier pictured in Figure 1, both in isolation and when combined into a pile.…”

“…In a similar regard to general clutter grasping, (Kessens et al, 2021) documents experiments relating to the grasping of a handle of a crate in order to utilize the affordance of opening; therefore, this paper only seeks to validate the grasp planner with regard to grasping the object class associated with the handle affordance prototype, shown in Figure 10 center. The ability of the pipeline to synthesize suitable grasps on the object specific prototype was analyzed using a dataset that was derived from experimental exercise in removing the test article with specific handle type, shown in Figure 11a, from within the container pictured in Figure 10 left.…”

“…Joint trajectories for gross end-effector motions were produced using the Search-based Motion Planing Library (SMPL) and MoveIt! as described in greater detail in the companion paper (Kessens et al, 2021) section 3.6.2, while inverse Jacobian Cartesian motions were computed through a software module termed the "Intrinsic Controller".…”

“…While the goal of the present paper is to discuss the design and performance of a robotic solution to T1, certain design choices represent a trade-off between constraints pertaining to T0. Those will be noted in the text, and the reader will be redirected to a second publication (Kessens et al, 2021) that provides a summarized discussion of all aspects of T0 jointly.…”