Task-Level Authoring for Remote Robot Teleoperation

Abstract

Remote teleoperation of robots can broaden the reach of domain specialists across a wide range of industries such as home maintenance, health care, light manufacturing, and construction. However, current direct control methods are impractical, and existing tools for programming robot remotely have focused on users with significant robotic experience. Extending robot remote programming to end users, i.e., users who are experts in a domain but novices in robotics, requires tools that balance the rich features necessary for complex teleoperation tasks with ease of use. The primary challenge to usability is that novice users are unable to specify complete and robust task plans to allow a robot to perform duties autonomously, particularly in highly variable environments. Our solution is to allow operators to specify shorter sequences of high-level commands, which we call task-level authoring, to create periods of variable robot autonomy. This approach allows inexperienced users to create robot behaviors in uncertain environments by interleaving exploration, specification of behaviors, and execution as separate steps. End users are able to break down the specification of tasks and adapt to the current needs of the interaction and environments, combining the reactivity of direct control to asynchronous operation. In this paper, we describe a prototype system contextualized in light manufacturing and its empirical validation in a user study where 18 participants with some programming experience were able to perform a variety of complex telemanipulation tasks with little training. Our results show that our approach allowed users to create flexible periods of autonomy and solve rich manipulation tasks. Furthermore, participants significantly preferred our system over comparative more direct interfaces, demonstrating the potential of our approach for enabling end users to effectively perform remote robot programming.

Augmented reality-assisted gesture-based teleoperated system for robot motion planning

“…As can be indicated from the quantitative and qualitative assessments of the current study, that the proposed HRI was able to combine the reactivity of the operator’s direct control with the powerful computer vision algorithms. It is worth noting that, similar studies like (Senft et al , 2021) which adopted semi-autonomous approach, demonstrated limited reactivity between the operator and the robot. As indicated in (Senft et al , 2021), the operator specifies a group of task-level steps without guiding the robot's motion throughout its path.…”

“…It is worth noting that, similar studies like (Senft et al , 2021) which adopted semi-autonomous approach, demonstrated limited reactivity between the operator and the robot. As indicated in (Senft et al , 2021), the operator specifies a group of task-level steps without guiding the robot's motion throughout its path. Operator’s guidance is essential in applications like radioactive isotopes production to avoid collision in crowded environment.…”

“…The questions used here are carefully designed to assess different aspects of system usability. Similar questions are commonly used in qualitative assessment of robotic systems (Senft et al, 2021;Ni et al, 2017;Fang et al, 2014) Regarding the previous knowledge of HRI, participants were divided into those that had no-to-little awareness of HRI (60%), and those who have a more in-depth understanding of the matter (40%), see Figure 16. Most of the participants referenced a goodto-optimum level of comfort while using the interaction equipment (90%).…”

“…It requires the operator’s undivided attention throughout the task working time which increases stress, especially when working in unstructured environments. Meanwhile, current studies based on a semiautonomous approach, such as Senft et al (2021), show a limited reactivity between the operator and the robot, as the operator specifies a group of task-level steps without guiding the robot’s motion throughout its path. This is obvious if compared to a full robot motion control in a gesture-based approach that mimics the operator’s arm motion.…”

“…In teleoperation, there is always concern about the suitable level of autonomy, especially when there is a delay and incomplete situational awareness (SA) (Yanco et al , 2015). Although the level of autonomy varies continuously from direct control method to fully sending a program asynchronously, however, there are three major categories (Senft et al , 2021): (1) teleprogramming, in which the operators develop and send programs that define the robot’s behaviors and responses to variations in its environment; (2) semiautonomous, where necessary, the human operator controls robot operations from time to time and can set higher-level activities that are carried out by the robot autonomously; and (3) direct control, in which all robot’s activities are manually controlled in real time by the operator.…”