Robotic Endoscope Control Via Autonomous Instrument Tracking

Abstract

Many keyhole interventions rely on bi-manual handling of surgical instruments, forcing the main surgeon to rely on a second surgeon to act as a camera assistant. In addition to the burden of excessively involving surgical staff, this may lead to reduced image stability, increased task completion time and sometimes errors due to the monotony of the task. Robotic endoscope holders, controlled by a set of basic instructions, have been proposed as an alternative, but their unnatural handling may increase the cognitive load of the (solo) surgeon, which hinders their clinical acceptance. More seamless integration in the surgical workflow would be achieved if robotic endoscope holders collaborated with the operating surgeon via semantically rich instructions that closely resemble instructions that would otherwise be issued to a human camera assistant, such as “focus on my right-hand instrument.” As a proof of concept, this paper presents a novel system that paves the way towards a synergistic interaction between surgeons and robotic endoscope holders. The proposed platform allows the surgeon to perform a bimanual coordination and navigation task, while a robotic arm autonomously performs the endoscope positioning tasks. Within our system, we propose a novel tooltip localization method based on surgical tool segmentation and a novel visual servoing approach that ensures smooth and appropriate motion of the endoscope camera. We validate our vision pipeline and run a user study of this system. The clinical relevance of the study is ensured through the use of a laparoscopic exercise validated by the European Academy of Gynaecological Surgery which involves bi-manual coordination and navigation. Successful application of our proposed system provides a promising starting point towards broader clinical adoption of robotic endoscope holders.

Towards Exoscope Automation in Neurosurgery: A Markerless Visual-Servoing Approach

“…Moreover, optical tracking systems require modification of the tracked instrument to which markers can be attached. This reduces the manageability of the instrument and can result in higher costs [16]. A marker-based approach for the automation of the da Vinci endoscope is proposed by [17].…”

“…The reconstructed position of the point of interest can eventually be used as feedback in the control loop of a robotic manipulator. This approach was used in the laparoscopy field by [20]. Here, a tooltip localisation method based on surgical tool segmentation and a visual-servoing approach was proposed.…”

“…Here, the potential of visual servoing in the medical field was demonstrated despite the simplified tracking techniques used. In the field of endoscopy, an autonomous system was proposed by [22] where only the endoscope was used as a vision sensor to segment and track surgical instruments, and a visual servo approach ensured smooth and appropriate movements of the endoscope.…”

Roadmap to Autonomous Surgery -- A Framework to Surgical Autonomy

“…More detailed checkpoints are required to measure the accuracy metrics for different surgery tasks [30]. What cannot be measured cannot be improved on.…”

Envisioning Robotic Exoscope: Concept and Preliminary Results

“…The continuous switching from operation theatre and visualization system reduces the smoothness of the surgical procedure, and the steep learning curve of the foot pedal [2] leads to switching to the conventional microscope in most cases. Visual servoing techniques [3] have been primarily studied in the context of camera automation in minimally invasive laparoscopy [4]. Its potential has become even more significant in the context of neurosurgery, as the only solution for tracking instruments is to employ vision sensors.…”