Whole-body multi-modal semi-autonomous teleoperation of mobile manipulator systems

Abstract

We propose a novel whole-body multi-modal semiautonomous teleoperation framework for mobile manipulator systems, which consists of: 1) Motion capture and wholebody motion mapping to allow the operator to intuitively teleoperate the mobile manipulator without being constrained by the master interface while also fully exploiting whole-body dexterity; 2) Slave robot autonomous control to allow the mobile manipulator to optimally track the operator's whole-body command, while taking into account the user-slave kinematic dissimilarity (slave robot's joint limit, joint velocity limit, and singularity); and 3) Visuo-haptic-vestibular feedback with HMD (Head Mounted Display) for 3D visual information, wearable cutaneous haptic device for manipulation force feedback, and actuated chair for vestibular feedback to reduce HMD-induced motion sickness. Performance of the proposed framework is validated with simulation of a ROV (remotely operated vehicle) manipulator system and some preliminary user studies.

Clustering user preferences for personalized teleoperation control schemes via trajectory similarity analysis

“…On the other hand, a master/puppet system is useful for creating an intuitive mapping between the local, master device and the remote puppet one (Glover et al, 2009;Park et al, 2021), although if there are any differences between the kinematics of the master and the puppet, a control scheme must be invented to translate between the two (Rebelo and Schiele, 2012). The final method, with the human-as-the-master, is similar to the master/puppet system but uses the human operator's own body as the master device, tracking its movement via cameras, datagloves, or IMUs (Ha et al, 2015;Jang et al, 2019). In this case, control schemes must be invented to translate the operator's motion into the joint space of the robot (Speeter, 1992;Rakita et al, 2017).…”

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

“…While using several camera perspectives may appear to be a solution, it results in an increment influx of information for the operator. The continuous processing and screening for helpful information in many video feeds places a cognitive burden on the user, resulting in inefficiencies and a reliance on user experience and technique [15] [16]. Constructing a 3D model of the robot within a virtual environment has the capability to increase user comprehension of the robot's status with a strong emphasis on depth perception [17].…”