Remote Robot Control with Human-in-the-Loop over Long Distances Using Digital Twins

Tsokalo, Ievgenii; Kapp, David; Kharabet, Ievgen; Fitzek, Frank H. P.; Reisslein, Martin

doi:10.1109/globecom38437.2019.9013428

Cited by 27 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The light covers a distance of 300 km per ms. For tele-healthcare applications between America and Europe with, e.g., a 9000-km distance, this would already be 30 ms and, thus, be too large for real-time applications. To enable tele-healthcare applications between greater distances, models also referred to as digital twins [ 35 ] are created from the operator or the patient and held as virtual objects in the communication network itself, in the so-called multi-access edge cloud. These virtual objects can then be calculated in the vicinity of the respective counterparts, thus virtually reducing the distance d .…”

Section: The Tactile Internetmentioning

confidence: 99%

“…Consider a scenario where remote patient connectivity is required to combine real-time care while removing the need to travel to remote locations, effectively increasing the number of patients that could directly be treated, even with human caregivers. Combining the different needs for the interaction of the tactile, auditory, and visual senses poses three different challenges with respect to bandwidth and latency, additionally limiting the geographical distance that can be covered unless special efforts are taken [ 35 ]. For example, the Cyberglove-II from CyberGlove Systems ( , accessed on 29 November 2021) employs 22 sensors and generates 1 byte of measurement data per sensor every 10 ms, which places the time resolution below the 1-ms threshold needed to provide fully immersive real-time experiences.…”

Section: Tele-healthcare Latency In the Real Worldmentioning

confidence: 99%

See 1 more Smart Citation

Healing Hands: The Tactile Internet in Future Tele-Healthcare

Senk

Ulbricht

Tsokalo³

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

In the early 2020s, the coronavirus pandemic brought the notion of remotely connected care to the general population across the globe. Oftentimes, the timely provisioning of access to and the implementation of affordable care are drivers behind tele-healthcare initiatives. Tele-healthcare has already garnered significant momentum in research and implementations in the years preceding the worldwide challenge of 2020, supported by the emerging capabilities of communication networks. The Tactile Internet (TI) with human-in-the-loop is one of those developments, leading to the democratization of skills and expertise that will significantly impact the long-term developments of the provisioning of care. However, significant challenges remain that require today’s communication networks to adapt to support the ultra-low latency required. The resulting latency challenge necessitates trans-disciplinary research efforts combining psychophysiological as well as technological solutions to achieve one millisecond and below round-trip times. The objective of this paper is to provide an overview of the benefits enabled by solving this network latency reduction challenge by employing state-of-the-art Time-Sensitive Networking (TSN) devices in a testbed, realizing the service differentiation required for the multi-modal human-machine interface. With completely new types of services and use cases resulting from the TI, we describe the potential impacts on remote surgery and remote rehabilitation as examples, with a focus on the future of tele-healthcare in rural settings.

show abstract

Section: The Tactile Internetmentioning

confidence: 99%

Section: Tele-healthcare Latency In the Real Worldmentioning

confidence: 99%

Healing Hands: The Tactile Internet in Future Tele-Healthcare

Senk

Ulbricht

Tsokalo³

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The method enables a robot to successfully push the new objects, which have not been seen in the training process. In [14], a human-machine-human control loop is established by a Digital Twin (DT) framework, which can achieve the low-latency visual feedback and short system reaction times between the operator and the robot. In [15], a novel DT prototype is developed, which explores the potential of deploying human-computer interaction technology to key scenarios, such as remote surgery.…”

Section: Related Workmentioning

confidence: 99%

Toward Verifying the User of Motion-Controlled Robotic Arm Systems via the Robot Behavior

Huang

Meng

Deng

et al. 2022

IEEE Internet Things J.

View full text Add to dashboard Cite

Motion-controlled robotic arms allow a user to interact with a remote real world without physically reaching it. By connecting cyberspace to the physical world, such interactive teleoperations are promising to improve remote education, virtual social interactions and online participatory activities. In this work, we build up a motion-controlled robotic arm framework comprising a robotic arm end and a user end, which are connected via a network and responsible for manipulator control and motion capture respectively. To protect the system access, we propose to verify who is controlling the robotic arm by examining the robotic arm's behavior, which adds a second security layer in addition to the system login credentials. We show that a robotic arm's motion inherits its human controller's behavioral biometric in interactive control scenarios. By extracting the angle readings of the robotic arm's all joints, the proposed user authentication approach reconstructs the robotic arm's end-effector movement trajectory that follows the user's hand. Furthermore, we derive the unique robotic motion features to capture the user's behavioral biometric embedded in the robot motions and develop learning-based algorithms to verify the robotic arm user to be one of the enrolled users or a nonuser. Extensive experiments show that our system achieves 94% accuracy to distinguish users while preventing user identity spoofing attacks with 95% accuracy.

show abstract

“…Ref. [37] presents a system architecture and controller that falls under the cyber-physical model by utilizing a digital twin of the robot. The idea here is that there are three control loops, one for the operator input, one for the physical robot, and one that serves as the interaction between the digital twin and the actual robot.…”

Section: Robot Control and Planningmentioning

confidence: 99%

A Systematic Review of Virtual Reality Interfaces for Controlling and Interacting with Robots

Wonsick

Padır

2020

Applied Sciences

View full text Add to dashboard Cite

There is a significant amount of synergy between virtual reality (VR) and the field of robotics. However, it has only been in approximately the past five years that commercial immersive VR devices have been available to developers. This new availability has led to a rapid increase in research using VR devices in the field of robotics, especially in the development of VR interfaces for operating robots. In this paper, we present a systematic review on VR interfaces for robot operation that utilize commercially available immersive VR devices. A total of 41 papers published between 2016–2020 were collected for review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Papers are discussed and categorized into five categories: (1) Visualization, which focuses on displaying data or information to operators; (2) Robot Control and Planning, which focuses on connecting human input or movement to robot movement; (3) Interaction, which focuses on the development of new interaction techniques and/or identifying best interaction practices; (4) Usability, which focuses on user experiences of VR interfaces; and (5) Infrastructure, which focuses on system architectures or software to support connecting VR and robots for interface development. Additionally, we provide future directions to continue development in VR interfaces for operating robots.

show abstract

Remote Robot Control with Human-in-the-Loop over Long Distances Using Digital Twins

Cited by 27 publications

References 12 publications

Healing Hands: The Tactile Internet in Future Tele-Healthcare

Healing Hands: The Tactile Internet in Future Tele-Healthcare

Toward Verifying the User of Motion-Controlled Robotic Arm Systems via the Robot Behavior

A Systematic Review of Virtual Reality Interfaces for Controlling and Interacting with Robots

Contact Info

Product

Resources

About