Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition

Steinmetz, Franz; Nitsch, Verena; Stulp, Freek

doi:10.1109/lra.2019.2928782

Cited by 40 publications

(27 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this confusion, for the final evaluated version of our tool, each Destination was manipulated directly, and an indicator showed when it was reachable by the robot Agent. However, for future versions of Authr, this capability may be added back in, for when users have a physical robot on the scene and wish to use the physical robot to configure the destinations and object locations more easily, much like interfaces such as Polyscope and RAZER [42]. to the goal location.…”

Section: Resultsmentioning

confidence: 99%

“…While the interface was successful in allowing users to specify complex programs, users had difficulties understanding the types and intentions of the robots' actions. RAZER was designed for task-level programming to allow shop-floor operators to leverage lowerlevel actions developed by experts, and it was later extended to support programming by demonstration [43,42]. They compared their solution with systems such as CoSTAR and Scratch, finding RAZER to be easier to understand by nonexperts.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Authr

Schoen

Henrichs

Strohkirch

et al. 2020

Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Authr

Schoen

Henrichs

Strohkirch

et al. 2020

Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

show abstract

“…Based on the insights of this work, we argue that adaptable haptically augmented VFs explain a task more accurately than oral or visual explanations. In the future, the tasks to be trained could be learned from expert demonstrations ( Steinmetz et al, 2019 ). To achieve this, an expert surgeon could perform a task first, allowing the abstraction of skills and corresponding support functions.…”

Section: Discussionmentioning

confidence: 99%

A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training

et al. 2021

Self Cite

View full text Add to dashboard Cite

Minimally invasive robotic surgery copes with some disadvantages for the surgeon of minimally invasive surgery while preserving the advantages for the patient. Most commercially available robotic systems are telemanipulated with haptic input devices. The exploitation of the haptics channel, e.g., by means of Virtual Fixtures, would allow for an individualized enhancement of surgical performance with contextual assistance. However, it remains an open field of research as it is non-trivial to estimate the task context itself during a surgery. In contrast, surgical training allows to abstract away from a real operation and thus makes it possible to model the task accurately. The presented approach exploits this fact to parameterize Virtual Fixtures during surgical training, proposing a Shared Control Parametrization Engine that retrieves procedural context information from a Digital Twin. This approach accelerates a proficient use of the robotic system for novice surgeons by augmenting the surgeon’s performance through haptic assistance. With this our aim is to reduce the required skill level and cognitive load of a surgeon performing minimally invasive robotic surgery. A pilot study is performed on the DLR MiroSurge system to evaluate the presented approach. The participants are tasked with two benchmark scenarios of surgical training. The execution of the benchmark scenarios requires basic skills as pick, place and path following. The evaluation of the pilot study shows the promising trend that novel users profit from the haptic augmentation during training of certain tasks.

show abstract

“…Current approaches differentiate between a low-level trajectories encoding and a high-level symbolic encoding. By monitoring and comparing the user's task demonstrations to pre-or postconditions of a predefined behavior set, a symbolic encoding partitions the task in action segments [11] [12]. In [8] this step is followed by arranging the segments of different task demonstrations in a generalized topology, while in [12], a recognized sequence of parameterized skills is propagated to a graphical interface for further processing by the user.…”

Section: Related Workmentioning

confidence: 99%

Collaborative Programming of Conditional Robot Tasks

Willibald

Eiband

Lee

2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Conventional robot programming methods are not suited for non-experts to intuitively teach robots new tasks. For this reason, the potential of collaborative robots for production cannot yet be fully exploited. In this work, we propose an active learning framework, in which the robot and the user collaborate to incrementally program a complex task. Starting with a basic model, the robot's task knowledge can be extended over time if new situations require additional skills. An on-line anomaly detection algorithm therefore automatically identifies new situations during task execution by monitoring the deviation between measured-and commanded sensor values. The robot then triggers a teaching phase, in which the user decides to either refine an existing skill or demonstrate a new skill. The different skills of a task are encoded in separate probabilistic models and structured in a high-level graph, guaranteeing robust execution and successful transition between skills. In the experiments, our approach is compared to two state-of-the-art Programming by Demonstration frameworks on a real system. Increased intuitiveness and task performance of the method can be shown, allowing shop-floor workers to program industrial tasks with our framework.

show abstract

Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition

Cited by 40 publications

References 21 publications

Authr

Authr

A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training

Collaborative Programming of Conditional Robot Tasks

Contact Info

Product

Resources

About