Autonomy infused teleoperation with application to brain computer interface controlled manipulation

Muelling, Katharina; Venkatraman, Arun; Valois, Jean-Sebastien; Downey, John E.; Weiss, Jeffrey M.; Javdani, Shervin; Hebert, Martial; Schwartz, Andrew B.; Collinger, Jennifer L.; Bagnell, J. Andrew

doi:10.1007/s10514-017-9622-4

Cited by 90 publications

(122 citation statements)

References 56 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…Shared control in assistive systems aims to reduce the user's cognitive and physical burden during task execution, typically without having the user relinquish complete control [3], [4], [5], [6]. In order to offset the drop in task performance due to shifting focus from the task at hand to switching between different control modes, various mode switch assistance paradigms have been proposed.…”

Section: Related Workmentioning

confidence: 99%

Active Intent Disambiguation for Shared Control Robots

Gopinath

Argall

2020

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Assistive shared-control robots have the potential to transform the lives of millions of people afflicted with severe motor impairments. The usefulness of shared-control robots typically relies on the underlying autonomy's ability to infer the user's needs and intentions, and the ability to do so unambiguously is often a limiting factor for providing appropriate assistance confidently and accurately. The contributions of this paper are four-fold. First, we introduce the idea of intent disambiguation via control mode selection, and present a mathematical formalism for the same. Second, we develop a control mode selection algorithm which selects the control mode in which the user-initiated motion helps the autonomy to maximally disambiguate user intent. Third, we present a pilot study with eight subjects to evaluate the efficacy of the disambiguation algorithm. Our results suggest that the disambiguation system (a) helps to significantly reduce task effort, as measured by number of button presses, and (b) is of greater utility for more limited control interfaces and more complex tasks. We also observe that (c) subjects demonstrated a wide range of disambiguation request behaviors, with the common thread of concentrating requests early in the execution. As our last contribution, we introduce a novel field-theoretic approach to intent inference inspired by dynamic field theory that works in tandem with the disambiguation scheme.

show abstract

Section: Related Workmentioning

confidence: 99%

Active Intent Disambiguation for Shared Control Robots

Gopinath

Argall

2020

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

“…As shown recently by Marturi et al [8] and earlier by Leeper et al [9], the task efficiency and accuracy are improved by incorporating further interfaced and autonomous functionalities. Muelling et al [10], for example, developed an integrated system of computer vision with manipulation capabilities, in which known objects with simple geometries are recognized, localized and grasped using depth images. Peer et al [11] and Salvietti et al [12] present telemanipulation approaches by mapping operator hand configurations to the robot hand and provide force feedback.…”

Section: Configurationsmentioning

confidence: 99%

Supervised Autonomous Locomotion and Manipulation for Disaster Response with a Centaur-Like Robot

Klamt

Rodríguez

Schwarz

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Mobile manipulation tasks are one of the key challenges in the field of search and rescue (SAR) robotics requiring robots with flexible locomotion and manipulation abilities. Since the tasks are mostly unknown in advance, the robot has to adapt to a wide variety of terrains and workspaces during a mission. The centaur-like robot Centauro has a hybrid legged-wheeled base and an anthropomorphic upper body to carry out complex tasks in environments too dangerous for humans. Due to its high number of degrees of freedom, controlling the robot with direct teleoperation approaches is challenging and exhausting. Supervised autonomy approaches are promising to increase quality and speed of control while keeping the flexibility to solve unknown tasks. We developed a set of operator assistance functionalities with different levels of autonomy to control the robot for challenging locomotion and manipulation tasks. The integrated system was evaluated in disaster response scenarios and showed promising performance. 2 https://newsroom.toyota.co.jp/en/download/ 20110424 arXiv:1809.06802v2 [cs.RO]

show abstract

“…Most recently, the work of Muelling et al . [8] presents a shared-control approach in the field of assistive robotics based on an invasive BCI. This is contrary to our approach which relies on a non-invasive BCI.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning Based BCI Control of a Robotic Service Assistant Using Intelligent Goal Formulation

Kuhner

Fiederer

Aldinger

et al. 2018

Preprint

View full text Add to dashboard Cite

As autonomous service robots become more affordable and thus available for the general public, there is a growing need for user-friendly interfaces to control these systems. Control interfaces typically get more complicated with increasing complexity of the robotic tasks and the environment. Traditional control modalities as touch, speech or gesture commands are not necessarily suited for all users. While non-expert users can make the effort to familiarize themselves with a robotic system, paralyzed users may not be capable of controlling such systems even though they need robotic assistance most. In this paper, we present a novel framework, that allows these users to interact with a robotic service assistant in a closed-loop fashion, using only thoughts. The system is composed of several interacting components: non-invasive neuronal signal recording and co-adaptive deep learning which form the brain-computer interface (BCI), high-level task planning based on referring expressions, navigation and manipulation planning as well as environmental perception. We extensively evaluate the BCI in various tasks, determine the performance of the goal formulation user interface and investigate its intuitiveness in a user study. Furthermore, we demonstrate the applicability and robustness of the system in real world scenarios, considering fetch-and-carry tasks and tasks involving human-robot interaction. As our results show, the system is capable of adapting to frequent changes in the environment and reliably accomplishes given tasks within a reasonable amount of time. Combined with high-level planning using referring expressions and autonomous robotic systems, interesting new perspectives open up for non-invasive BCI-based human-robot interactions.

show abstract

Autonomy infused teleoperation with application to brain computer interface controlled manipulation

Cited by 90 publications

References 56 publications

Active Intent Disambiguation for Shared Control Robots

Active Intent Disambiguation for Shared Control Robots

Supervised Autonomous Locomotion and Manipulation for Disaster Response with a Centaur-Like Robot

Deep Learning Based BCI Control of a Robotic Service Assistant Using Intelligent Goal Formulation

Contact Info

Product

Resources

About