Robust shared autonomy for mobile manipulation with continuous scene monitoring

Abstract

This work presents a fully integrated system for reliable grasping and manipulation using dense visual mapping, collision-free motion planning, and shared autonomy. The particular motion sequences are composed automatically based on high-level objectives provided by a human operator, with continuous scene monitoring during execution automatically detecting and adapting to dynamic changes of the environment. The system is able to automatically recover from a variety of disturbances and fall back to the operator if stuck or if it cannot otherwise guarantee safety. Furthermore, the operator can take control at any time and then resume autonomous operation. Our system is flexible to be adapted to new robotic systems, and we demonstrate our work on two real-world platformsfixed and floating base -in shared workspace scenarios.To the best of our knowledge, this work is also the first to employ the inverse Dynamic Reachability Map for realtime, optimized mobile base positioning to maximize workspace manipulability reducing cycle time and increasing planning and autonomy robustness.

Towards Shared Autonomy Applications using Whole-body Control Formulations of Locomanipulation

“…We have formulated the tracking problem as unconstrained minimization of the end-effector position in the global frame over the base position and the arm configuration and used the Levenberg-Marquardt [23] algorithm to solve this problem. Note, this is a relaxation of Equations (1)- (4) as the manipulator may pass through singular configurations resulting in a violation of real-time requirements. The results are depicted in Figure 4 validating the relaxation to be suitable, and snapshots of an applicable real-world task experiment depicted in Figure 6.…”

“…However, due to the complexity of planning locomanipulation in real-time, locomotion/navigation and manipulation are often treated as separate problems and joined and coordinated by a high-level state machine [2], sequence planner, or shared autonomy control interface [3], [4]. In this case, the problems of optimal base placement [5], navigation to the base placement, and fixed-base manipulation [6], [7] are treated separately, limiting the applicability to static targets and obstacles and disregarding the inherent redundancy of high degree of freedom (DoF) systems.…”

Planning in Time-Configuration Space for Efficient Pick-and-Place in Non-Static Environments with Temporal Constraints

“…Such artifacts which are common to geometric sampling-based planning methods point out possible future work in extending the proposed method to kinodynamic planning or to incorporate it with trajectory optimization for generating smooth and optimal motion. Finally, an integration with a shared autonomy system such as [24] opens up the possibility for collaborative mobile manipulation applications and deployments.…”

HDRM: A Resolution Complete Dynamic Roadmap for Real-Time Motion Planning in Complex Scenes

“…Both DRM and HDRM compute solutions in static environments, which can be different between planning queries but need to be static during execution-they inherently do not adapt to runtime changes. Since HDRM guarantees resolution completeness and is able to plan in real-time (few milliseconds or less), the future work will focus on implementing a closedloop online adaptation/re-planning framework for applications such as real-time interaction between human and robot in a shared workspace similar to [24].…”