Still Not Solved: A Call for Renewed Focus on User-Centered Teleoperation Interfaces

Abstract

Teleoperation is one of the oldest applications of human-robot interaction, yet decades later, robots are still difficult to control in a variety of situations, especially when used by non-expert robot operators. That difficulty has relegated teleoperation to mostly expert-level use cases, though everyday jobs and lives could benefit from teleoperated robots by enabling people to get tasks done remotely. Research has made great progress by improving the capabilities of robots, and exploring a variety of interfaces to improve operator performance, but many non-expert applications of teleoperation are limited by the operator’s ability to understand and control the robot effectively. We discuss the state of the art of user-centered research for teleoperation interfaces along with challenges teleoperation researchers face and discuss how an increased focus on human-centered teleoperation research can help push teleoperation into more everyday situations.

A Systematic Comparison of Simulation Software for Robotic Arm Manipulation using ROS2

“…Our experimental results show that TELESIM is modular, plug-and-play and user-friendly, as not only were we able to deploy it on 2 robots with different modalities, but most users were able to succeed by building at least once a tower of 3 cubes, with only 5 minutes of training, regardless of the control modality or robot used. We thus bridged the gap pointed out by Rea & Seo [1], where they state that there is a lack of non-expert evaluation of robotic teleoperation for general tasks such as picking and placing common objects. TELESIM is available on GitHub 1 , allowing developers to perform teleoperation on their robots with minimal setup time.…”

“…Robot teleoperation is difficult for non-experts [1], [2]. Recently, the ANA Avatar XPRIZE Challenge [3] set a series of challenging tasks to test the limits of teleoperation.…”

“…Although this paper focuses on TELESIM as a framework, our evaluation also addresses two main limitations of previous work: (1) researchers have only used one robot per study, and (2) most user studies consider a small user base, which does not represent a statistically significant sample. It also addresses a gap discussed by Rea & Seo [1], which states that there needs to be more non-expert evaluation of robotic teleoperation for general tasks such as picking and placing common objects. Therefore, to advance the state-of-the-art in robotic teleoperation, we investigate the performance of different control modalities for direct teleoperation using a VR controller and finger mapping.…”

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

“…Assistive machines range from low control complexity, such as mechanical wheelchairs, to high complexity, such as robotic arms. Robotic arms with a large number of degrees of freedom (DoFs) can be difficult to control with current commercially-available interfaces [10]. A common solution to this mismatch in dimensionality is modal control [11], in which only a subset of the control dimensions of the robot are operated at a given time.…”

Clustering user preferences for personalized teleoperation control schemes via trajectory similarity analysis

“…The control scheme design challenge was reaffirmed as unsolved as recently as 2022 ( Rea and Seo, 2022 ), in which paper the authors suggest that improvements from user-centered design have trailed improvements in technology, creating a situation in which the main factor limiting teleoperated robot usage is the difficulty of learning to effectively control them. A common approach to control scheme design is for a researcher to design several mappings and then compare user performance for each control scheme ( Wang et al, 2018 ; Hetrick et al, 2020 ; Whitney et al, 2020 ; Meeker et al, 2022 ; Molnar and Mengüç, 2022 ).…”