A new framework for vision-enabled and robotically assisted minimally invasive surgery

Abstract

This paper presents our on-going research at bringing the state-of-the-art in vision and robotics technologies to enhance the emerging minimally invasive surgery, in particular the laparoscopic surgical procedure (Figure 1). A f r amework that utilizes intelligent visual modeling, recognition, and servoing capabilities for assisting the surgeon in maneuvering the scope (camera) in laparoscopy is proposed.The proposed framework integrates top-down model guidance, bottom-up image analysis, and surgeon-in-the-loop monitoring for added patient safety. For the top-down directives, high-level models are used to represent the abdominal anatomy and to encode choreographed scope movement sequences based on the surgeon's knowledge. For the bottom-up analysis, vision algorithms are designed for image analysis, modeling, and matching in a exible, deformable environment (the abdominal cavity). For reconciling the top-down and bottom-up activities, robot servoing mechanisms are realized for executing choreographed scope movements with active vision guidance.The proposed choreographed scope maneuvering concept facilitates the surgeon's control of his/her visual feedback in a handless manner, reduces the risk to the patient from inappropriate scope movements by an assistant, and allows the operation to be p erformed faster and with greater ease. In this paper, we describe the new framework and present some preliminary results on laparoscopic image analysis for segmentation and instrument localization, and on instrument tracking.

Video-based assistance system for training in minimally invasive surgery

“…The system is not affected by the specular reflections as other approaches implement some methods to solve this problem (17,22,26).…”

“…We think that addressing the problem by using image processing techniques to track the instrument is an interesting alternative to others based on the use of sensors located on the instrument (optical, electromagnetic or mechanical) which can be very uncomfortable and unmanageable for surgeons. In the scientific literature, some of these approaches analyze the colour of the instrument as Wang et al (17), who carry out a statistical analysis to distinguish between pixels belonging to a laparoscopic instrument or environment of the abdominal cavity. Other authors (18), however, perform statistical analysis using Bayes theorem.…”

Navigating inner space: 3-D assistance for minimally invasive surgery

“…Successful experiments using this system were done on living pigs. Wang and Ueker [10] describe a framework that utilizes intelligent visual modeling, recognition, and servoing methods to assist the surgeon in manipulating a laparoscope. It integrates top-down model guidance, bottom-up image analysis, and surgeonin-the-loop monitoring.…”

How Does the Camera Assistant Decide the Zooming Ratio of Laparoscopic Images? Analysis and Implementation

“…To free the surgeon from the task of controlling the view and to automatically offer an optimal and stable view during laparoscopic surgery, several automatic camera positioning systems have been devised [1,2,3,4,5,6]. These systems visually extract the shape and/or position of the surgical instrument from the laparoscopic images in real time, and automatically manipulate the laparoscope to always center the tip of the instrument in the displayed image.…”

“…The best zooming ratio depends on both the surgical procedure/phase and the habits/preferences of the operating surgeon. For this reason, most of the instrument tracking systems recently developed [3,4,5,6] have abandoned the idea of systematic control of zooming parameters; instead, the surgeon is required to define the parameters preoperatively or adjust them intraoperatively through conventional human-machine interfaces, which again means an extra control burden for the surgeon.…”