Concept of a Mixed-Reality Learning Environment for Collaborative Robotics

Sievers, Torsten; Schmitt, Bianca; Rückert, Patrick; Petersen, Maren; Tracht, Kirsten

doi:10.1016/j.promfg.2020.04.034

Cited by 21 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mixed reality (MR) technology involves the merging of physical and imaginary spaces and does not occur exclusively in the physical or virtual world [90]. MR enables real-time visualization and interaction between physical and digital content [91] and takes full advantage of the visual information of real scenes and the three-dimensional immersion and interaction provided with virtual cues [92].…”

Section: Reality-virtuality Continuummentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios.

show abstract

Section: Reality-virtuality Continuummentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…visual programming with skills [10] or learning from demonstration [61]), the more general term teaching time D teach will be used hereinafter. It is important to notice that this term does not necessarily have to refer to the teaching of robots only: safe and efficient co-working is supported by well-trained employees [62], and qualification times can thus additionally be taken into account when measuring D teach . This absolute metric is hard to compare across tasks.…”

Section: B Flexibilitymentioning

confidence: 99%

Benchmarking Teamwork of Humans and Cobots—An Overview of Metrics, Strategies, and Tasks

2023

View full text Add to dashboard Cite

Human-robot teaming receives an ever-increasing level of attention in research, development and industry. Novel approaches to task sharing in hybrid teams range from optimized schedules to intelligent cobot assistants with a high degree of autonomy. These approaches must prove their usefulness and benefits compared to manual work or full automation -particularly when it comes to assessing their potential for productive industrial use. This leverages demand for standardized, repeatable benchmarks to compare approaches and measure improvements in a reproducible way. Designing such benchmarks is challenging as numerous aspects, from safety considerations to human factors and team performance, must be considered. This survey seeks to contribute to the future development of benchmarks for the field of collaborative assembly, handling, and industrial cobot applications by giving a comprehensive overview of relevant metrics, evaluation strategies, and tasks for human-robot teams.

show abstract

“…The system aimed to simulate its behavior using the formulated DT model. Another DT-based virtual learning environment for collaborative assembly tasks was presented in [36]. It aimed to train unskilled employees to collaborate with robots.…”

Section: Digital Twins In Industry 40mentioning

confidence: 99%

Digital Twin for Human–Robot Interactions by Means of Industry 4.0 Enabling Technologies

Gallala

Kumar

Hichri

et al. 2022

Sensors

View full text Add to dashboard Cite

There has been a rapid increase in the use of collaborative robots in manufacturing industries within the context of Industry 4.0 and smart factories. The existing human–robot interactions, simulations, and robot programming methods do not fit into these fast-paced technological advances as they are time-consuming, require engineering expertise, waste a lot of time in programming and the interaction is not trivial for non-expert operators. To tackle these challenges, we propose a digital twin (DT) approach for human–robot interactions (HRIs) in hybrid teams in this paper. We achieved this using Industry 4.0 enabling technologies, such as mixed reality, the Internet of Things, collaborative robots, and artificial intelligence. We present a use case scenario of the proposed method using Microsoft Hololens 2 and KUKA IIWA collaborative robot. The obtained results indicated that it is possible to achieve efficient human–robot interactions using these advanced technologies, even with operators who have not been trained in programming. The proposed method has further benefits, such as real-time simulation in natural environments and flexible system integration to incorporate new devices (e.g., robots or software capabilities).

show abstract

Concept of a Mixed-Reality Learning Environment for Collaborative Robotics

Cited by 21 publications

References 14 publications

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Benchmarking Teamwork of Humans and Cobots—An Overview of Metrics, Strategies, and Tasks

Digital Twin for Human–Robot Interactions by Means of Industry 4.0 Enabling Technologies

Contact Info

Product

Resources

About