Automated instrument tracking in robotically assisted laparoscopic surgery

Abstract

This paper describes a practical and reliable image analysis and tracking algorithm to achieve automated instrument localization and scope maneuvering in robotically assisted laparoscopic surgery. Laparoscopy is a minimally invasive surgical procedure that utilizes multiple small incisions on the patient's body through which the surgeon inserts tools and a videoscope in order to conduct an operation. The scope relays images of internal organs to a camera, and the images are displayed on a video screen. The surgeon performs the operation by viewing the scope images rather than performing the traditional “open” procedure, where a large incision is made on the patient's body for direct viewing. The current mode of laparoscopy employs an assistant to hold the scope and position it in response to the surgeon's verbal commands. However, this results in suboptimal visual feedback, because the scope is often aimed incorrectly and vibrates due to hand trembling. We have developed a robotic laparoscope positioner to replace the assistant. The surgeon commands the robotic positioner through a hand/foot controller interface. To further simplify the human‐machine interface that controls the robotic scope positioner, we report here a novel scope‐positioning scheme using automated image analysis and robotic visual servoing. The scheme enables the surgeon to control visual feedback and to perform surgery more efficiently without requiring additional use of the hands. J Image Guid Surg 1:308–325 (1995). © 1996 Wiley‐Liss, Inc.

Product of tracking experts for visual tracking of surgical tools

“…Other approaches (that do not necessarily modify the tool itself) include: using color space for classification of pixel into instruments and organs, performing shape analysis of these classification labels and then predicting the location of tool in next frame using an Auto Regressive Model [23].…”

Surgical Vision

“…In robotic surgery systems, visual servoing requires vision-based instrument tracking and a system for simplifying camera control was shown with the development of the AESOP robotic laparoscope holder. 122 The proposed visual instrument tracking method was based on the probabilistic classification of image pixels based on their color using a Bayesian filter. This was followed by clustering, cluster analysis and temporal filtering.…”

Surgical and interventional robotics: part III [Tutorial]

“…This presents another opportunity for robotic assistance, especially when computer control can improve the efficiency or quality of assistance. Some of the earliest medical robots, such as Aesop [49] and LARS [47], were camera holders for minimally invasive procedures. In these systems, the surgeon used an alternate interface, such as a foot pedal or speech, to command the robot to reposition the camera.…”