Interactive Planning and Supervised Execution for High-Risk, High-Latency Teleoperation

Pryor, Will; Vágvölgyi, Balázs; Deguet, Anton; Léonard, Simon; Whitcomb, Louis L.; Kazanzides, Peter

doi:10.1109/iros45743.2020.9340800

Cited by 13 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All user studies were approved by the Johns Hopkins University Homewood Institutional Review Board (protocol HIRB00000701). The studies are reported in chronological order, beginning with model-mediated teleoperation experiments from Vozar et al (2015a , b) , followed by augmented virtuality experiments from Vagvolgyi et al (2017 , 2018) ; Pryor et al (2019) , and then IPSE experiments from Pryor et al (2020) . We do not report test results for individual components, such as the task monitors or model update methods, which can be found in the relevant cited papers.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Teleoperation and Visualization Interfaces for Remote Intervention in Space

et al. 2021

Self Cite

View full text Add to dashboard Cite

Approaches to robotic manufacturing, assembly, and servicing of in-space assets range from autonomous operation to direct teleoperation, with many forms of semi-autonomous teleoperation in between. Because most approaches require one or more human operators at some level, it is important to explore the control and visualization interfaces available to those operators, taking into account the challenges due to significant telemetry time delay. We consider one motivating application of remote teleoperation, which is ground-based control of a robot on-orbit for satellite servicing. This paper presents a model-based architecture that: 1) improves visualization and situation awareness, 2) enables more effective human/robot interaction and control, and 3) detects task failures based on anomalous sensor feedback. We illustrate elements of the architecture by drawing on 10 years of our research in this area. The paper further reports the results of several multi-user experiments to evaluate the model-based architecture, on ground-based test platforms, for satellite servicing tasks subject to round-trip communication latencies of several seconds. The most significant performance gains were obtained by enhancing the operators’ situation awareness via improved visualization and by enabling them to precisely specify intended motion. In contrast, changes to the control interface, including model-mediated control or an immersive 3D environment, often reduced the reported task load but did not significantly improve task performance. Considering the challenges of fully autonomous intervention, we expect that some form of teleoperation will continue to be necessary for robotic in-situ servicing, assembly, and manufacturing tasks for the foreseeable future. We propose that effective teleoperation can be enabled by modeling the remote environment, providing operators with a fused view of the real environment and virtual model, and incorporating interfaces and control strategies that enable interactive planning, precise operation, and prompt detection of errors.

show abstract

Section: Resultsmentioning

confidence: 99%

“…The IPSE module was evaluated in the Soft-MLI setup, with the mock servicing robot and satellite shown in Figure 13A , Pryor et al (2020) . Each experiment began with the environment model already created, as described in Section 2.1.1 .…”

Section: Resultsmentioning

confidence: 99%

Teleoperation and Visualization Interfaces for Remote Intervention in Space

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results of these interviews highlight the need for operator-centric approaches to demonstrate system trustworthiness, across telemanipulation applications. Examples of technical solutions suggested in the literature (Artigas et al, 2016 ; Panzirsch et al, 2020 ; Pryor et al, 2020 ) may be valuable in addressing the challenges posed by delayed telemanipulation, but in order to be used confidently, their capabilities must be gradually demonstrated to operators in scenarios with gradually increasing risk. Furthermore, new features should be explained to operators so that they understand their functionality, capabilities, and limitations.…”

Section: Discussionmentioning

confidence: 99%

“…The literature suggests various technical approaches which could improve performance of delayed telemanipulation systems (Beik-Mohammadi et al, 2020 ; Panzirsch et al, 2020 ; Pryor et al, 2020 ) but there has been limited other research investigating whether any of these features demonstrate trustworthiness Rogers et al ( 2017 ). As trust is a human quality and, therefore, its perception depends on the human user of a system, user-centric and non-technical approaches are necessary for its development.…”

Section: Introductionmentioning

confidence: 99%

Elicitation of trustworthiness requirements for highly dexterous teleoperation systems with signal latency

Louca,

Vrublevskis,

Eder

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

IntroductionTeleoperated robotic manipulators allow us to bring human dexterity and cognition to hard-to-reach places on Earth and in space. In long-distance teleoperation, however, the limits of the speed of light results in an unavoidable and perceivable signal delay. The resultant disconnect between command, action, and feedback means that systems often behave unexpectedly, reducing operators' trust in their systems. If we are to widely adopt telemanipulation technology in high-latency applications, we must identify and specify what would make these systems trustworthy.MethodsIn this requirements elicitation study, we present the results of 13 interviews with expert operators of remote machinery from four different application areas—nuclear reactor maintenance, robot-assisted surgery, underwater exploration, and ordnance disposal—exploring which features, techniques, or experiences lead them to trust their systems.ResultsWe found that across all applications, except for surgery, the top-priority requirement for developing trust is that operators must have a comprehensive engineering understanding of the systems' capabilities and limitations. The remaining requirements can be summarized into three areas: improving situational awareness, facilitating operator training, and familiarity, and easing the operator's cognitive load.DiscussionWhile the inclusion of technical features to assist the operators was welcomed, these were given lower priority than non-technical, user-centric approaches. The signal delays in the participants' systems ranged from none perceived to 1 min, and included examples of successful dexterous telemanipulation for maintenance tasks with a 2 s delay. As this is comparable to Earth-to-orbit and Earth-to-Moon delays, the requirements discussed could be transferable to telemanipulation tasks in space.

show abstract

“…While many 3D input devices have been developed, lowdimensional input devices such as joysticks and computer mice are still widely used in professional teleoperation systems such as surgical robots [20], assistive robots [21], disaster rescue robots [22], and space exploration robots [23]. Different kinds of 2D input devices have been used for a human operator to specify the position of a robot endeffector, including joysticks [24]- [27], computer mice [28], and touch screens [29], [30].…”

Section: Telemanipulation With Low-dimensional Input Devicesmentioning

confidence: 99%

A Design Space of Control Coordinate Systems in Telemanipulation

Wang,

Praveena,

Gleicher

2024

IEEE Access

View full text Add to dashboard Cite

Teleoperation systems map operator commands from an input device into some coordinate frame in the remote environment. This frame, which we call a control coordinate system, should be carefully chosen as it determines how operators should move to get desired robot motions. While specific choices made by individual systems have been described in prior work, a design space, i.e., an abstraction that encapsulates the range of possible options, has not been codified. In this paper, we articulate a design space of control coordinate systems, which can be defined by choosing a direction in the remote environment for each axis of the input device. Our key insight is that there is a small set of meaningful directions in the remote environment. Control coordinate systems in prior works can be organized by the alignments of their axes with these directions and new control coordinate systems can be designed by choosing from these directions. We also provide three design criteria to reason about the suitability of control coordinate systems for various scenarios. To demonstrate the utility of our design space, we use it to organize prior systems and design control coordinate systems for three scenarios that we assess through human-subject experiments. Our results highlight the promise of our design space as a conceptual tool to assist system designers to design control coordinate systems that are effective and intuitive for operators.

show abstract

Interactive Planning and Supervised Execution for High-Risk, High-Latency Teleoperation

Cited by 13 publications

References 13 publications

Teleoperation and Visualization Interfaces for Remote Intervention in Space

Teleoperation and Visualization Interfaces for Remote Intervention in Space

Elicitation of trustworthiness requirements for highly dexterous teleoperation systems with signal latency

A Design Space of Control Coordinate Systems in Telemanipulation

Contact Info

Product

Resources

About