Motion Polytopes in Virtual Reality for Shared Control in Remote Manipulation Applications

Abstract

In remote applications that mandate human supervision, shared control can prove vital by establishing a harmonious balance between the high-level cognition of a user and the low-level autonomy of a robot. Though in practice, achieving this balance is a challenging endeavor that largely depends on whether the operator effectively interprets the underlying shared control. Inspired by recent works on using immersive technologies to expose the internal shared control, we develop a virtual reality system to visually guide human-in-the-loop manipulation. Our implementation of shared control teleoperation employs end effector manipulability polytopes, which are geometrical constructs that embed joint limit and environmental constraints. These constructs capture a holistic view of the constrained manipulator’s motion and can thus be visually represented as feedback for users on their operable space of movement. To assess the efficacy of our proposed approach, we consider a teleoperation task where users manipulate a screwdriver attached to a robotic arm’s end effector. A pilot study with prospective operators is first conducted to discern which graphical cues and virtual reality setup are most preferable. Feedback from this study informs the final design of our virtual reality system, which is subsequently evaluated in the actual screwdriver teleoperation experiment. Our experimental findings support the utility of using polytopes for shared control teleoperation, but hint at the need for longer-term studies to garner their full benefits as virtual guides.

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

“…Thus, the user perception from 2D visual feedback makes it insufficient to accomplish remote tasks effectively. Hence, the success and efficiency of teleoperated tasks mainly depend on operator skill and proficiency [45,46].…”

Polytope-based Continuous Scalar Performance Measure with Analytical Gradient for Effective Robot Manipulation

“…The convex nature can be used in conjunction with geometric operations such as Minkowski sum or intersection to obtain capacities of composite chains [10]. Additionally, constrained motion polytopes [11], which consider both intrinsic and environmental constraints, can be used as visual aids and virtual guides in virtual reality for shared teleoperation [12]. With its algebraic representation, supplementary constraints such as proximity to danger zones [13], zero-moment points (ZMP) [14] or multi-contact humanoid locations [15] can be easily added.…”