Inferring Semantic State Transitions During Telerobotic Manipulation

Bauer, Adrian S.; Schmaus, Peter; Albu‐Schäffer, Alin; Leidner, Daniel

doi:10.1109/iros.2018.8594458

Cited by 9 publications

(9 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the robot slides or tilts an object during a motion, this cannot be registered. This would require a physical simulation of the world, as it is suggested in [29]. However, changes in the world triggered by a human worker, for instance, would still not be reflected in the world model.…”

Section: Discussionmentioning

confidence: 99%

Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition

Steinmetz

Nitsch

Stulp

2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition

Steinmetz

Nitsch

Stulp

2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

“…In our implementation, object detection and localization are performed according to Sundermeyer et al (2018). The symbolic state of the environment is evaluated based on a digital twin of the robot and the environment in simulation as described in Bauer et al (2018).…”

Section: Model Updatementioning

confidence: 99%

Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

et al. 2021

Self Cite

View full text Add to dashboard Cite

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges.

show abstract

“…This is the case when dealing with probabilistic action effects or in case of possible failures. [20] present a framework for synchronizing a physical simulation of the world with the real world state and extracting semantic representations from it. We use this framework for retrieving the semantic world state from simulations we run.…”

Section: Semantic State Inferencementioning

confidence: 99%

“…As we believe that probabilistic effects in manipulation tasks are induced by epistemic uncertainty, we introduce small errors at each step of the action in simulation. The scaling of the errors must be adapted to reproduce the probabilities that are already known from experience, e. g. using an evolutionary algorithm as in [20].…”

Section: Refining Predictions With Physical Simulationmentioning

confidence: 99%

See 1 more Smart Citation

Probabilistic Effect Prediction through Semantic Augmentation and Physical Simulation

Bauer

Schmaus

Stulp

et al. 2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Nowadays, robots are mechanically able to perform highly demanding tasks, where AI-based planning methods are used to schedule a sequence of actions that result in the desired effect. However, it is not always possible to know the exact outcome of an action in advance, as failure situations may occur at any time. To enhance failure tolerance, we propose to predict the effects of robot actions by augmenting collected experience with semantic knowledge and leveraging realistic physics simulations. That is, we consider semantic similarity of actions in order to predict outcome probabilities for previously unknown tasks. Furthermore, physical simulation is used to gather simulated experience that makes the approach robust even in extreme cases. We show how this concept is used to predict action success probabilities and how this information can be exploited throughout future planning trials. The concept is evaluated in a series of real world experiments conducted with the humanoid robot Rollin' Justin.

show abstract

Inferring Semantic State Transitions During Telerobotic Manipulation

Cited by 9 publications

References 26 publications

Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition

Intuitive Task-Level Programming by Demonstration Through Semantic Skill Recognition

Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

Probabilistic Effect Prediction through Semantic Augmentation and Physical Simulation

Contact Info

Product

Resources

About