A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training

Abstract

Minimally invasive robotic surgery copes with some disadvantages for the surgeon of minimally invasive surgery while preserving the advantages for the patient. Most commercially available robotic systems are telemanipulated with haptic input devices. The exploitation of the haptics channel, e.g., by means of Virtual Fixtures, would allow for an individualized enhancement of surgical performance with contextual assistance. However, it remains an open field of research as it is non-trivial to estimate the task context itself during a surgery. In contrast, surgical training allows to abstract away from a real operation and thus makes it possible to model the task accurately. The presented approach exploits this fact to parameterize Virtual Fixtures during surgical training, proposing a Shared Control Parametrization Engine that retrieves procedural context information from a Digital Twin. This approach accelerates a proficient use of the robotic system for novice surgeons by augmenting the surgeon’s performance through haptic assistance. With this our aim is to reduce the required skill level and cognitive load of a surgeon performing minimally invasive robotic surgery. A pilot study is performed on the DLR MiroSurge system to evaluate the presented approach. The participants are tasked with two benchmark scenarios of surgical training. The execution of the benchmark scenarios requires basic skills as pick, place and path following. The evaluation of the pilot study shows the promising trend that novel users profit from the haptic augmentation during training of certain tasks.

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

“…As evident from Fig. 6 , DT-aided medical telerobotics was one of the fastest growing areas in recent years, with its key applications in robotic surgeries [30] , [31] , [32] , [33] , [34] . Again, in substantial instances of the medical telerobotic approaches explored in this study utilized an immersive VR interface/environment [30] , [35] , [36] .…”

“… Enhanced spatial information. 2021 Hagmann et al [32] Haptic telerobotic system to aid trainee surgeons in performing minimally invasive surgeries with reduced cognitive burden, in which the Robot-centric DT replicated the robot motions while delivering required semantic information. 60% increase in accuracy for path following, decreased time consumption, and minimal deviation from allocated trajectories.…”

CATs: Task Planning for Shared Control of Assistive Robots with Variable Autonomy

“…SCTs are action representations that allow users to control high-dimensional robotic systems with only low-dimensional user input commands [2], [9], for instance from a 3D joystick or electromyography sensors. They provide task-relevant support in fields such as assistive robotics [2] or robotic surgery [14].…”

Multi-Phase Multi-Modal Haptic Teleoperation

“…Virtual Fixtures (VFs) enhance teleoperation by haptically guiding the human operator, or keeping the robot out of forbidden regions [1]- [3]. Most fixtures are based on a geometric definition and are statically linked to e.g.…”