Toward Detection and Localization of Instruments in Minimally Invasive Surgery

Abstract

Methods for detecting and localizing surgical instruments in laparoscopic images are an important element of advanced robotic and computer-assisted interventions. Robotic joint encoders and sensors integrated or mounted on the instrument can provide information about the tool's position, but this often has inaccuracy when transferred to the surgeon's point of view. Vision sensors are currently a promising approach for determining the position of instruments in the coordinate frame of the surgical camera. In this study, we propose a vision algorithm for localizing the instrument's pose in 3-D leaving only rotation in the axis of the tool's shaft as an ambiguity. We propose a probabilistic supervised classification method to detect pixels in laparoscopic images that belong to surgical tools. We then use the classifier output to initialize an energy minimization algorithm for estimating the pose of a prior 3-D model of the instrument within a level set framework. We show that the proposed method is robust against noise using simulated data and we perform quantitative validation of the algorithm compared to ground truth obtained using an optical tracker. Finally, we demonstrate the practical application of the technique on in vivo data from minimally invasive surgery with traditional laparoscopic and robotic instruments.

Fast Part-Based Classification for Instrument Detection in Minimally Invasive Surgery

“…A video demonstrating the functioning of this algorithm is available in the supplementary materials. Note that while this algorithm is used with a boosted classifier in this paper, combining it with RF classifiers as those in [4,5] is also possible.…”

“…Its goal is to provide accurate 2D or 3D location estimates of surgical instruments from visual data. Critical to a number of applications such as automatic endoscope control [1], instrument-surface detection [2], clinician training evaluation [3] or setting virtual constraints for instrument motion [4,5], instrument detection and tracking can significantly augment the clinicians experience during surgical procedures.…”

“…Among the many approaches proposed in the last twenty years [1,2,6,7,8,9], recent detection-based schemes that rely on building statistical classifiers to evaluate the presence of the instrument appear to be the most promising for in-vivo detection and tracking [4,5,10]. Within this last category of methods, Reiter et al [5] combined a multiclass Random Forest (RF) [11] labelling approach with robot kinematic information to estimate the instrument 3D pose.…”

“…Within this last category of methods, Reiter et al [5] combined a multiclass Random Forest (RF) [11] labelling approach with robot kinematic information to estimate the instrument 3D pose. In [4], RFs were also used to handle instrument-background classification, giving way to instrument segmentations and the 3D pose. Alternatively, [10] combined template tracking and binary classification to provide 2D instrument positions in retinal microsurgery.…”

“…For example, in [4,5], run-time classification is described as a computational bottleneck. As a result it is performed at a single predefined image scale and using only a limited number of RF trees.…”