Learning features on robotic surgical tools

Reiter, Austin; Allen, Peter K.; Zhao, Tao

doi:10.1109/cvprw.2012.6239245

Cited by 8 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is challenging for accurate absolute position sensing and requires time-consuming hand-eye calibration between the camera and the robot coordinates. On cable driven systems the absolute error can be up to 1 inch, which means the positional accuracy is potentially too low for tracking applications without visual correction [1] – [3] . Recent developments in endoscopic computer vision have resulted in advanced approaches for 2D instrument detection for minimally invasive surgery.…”

Section: Introductionmentioning

confidence: 99%

Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks

Kurmann

Chang³

et al. 2018

IEEE Trans. Med. Imaging

107

111

View full text Add to dashboard Cite

Instrument detection, pose estimation, and tracking in surgical videos are an important vision component for computer-assisted interventions. While significant advances have been made in recent years, articulation detection is still a major challenge. In this paper, we propose a deep neural network for articulated multi-instrument 2-D pose estimation, which is trained on detailed annotations of endoscopic and microscopic data sets. Our model is formed by a fully convolutional detection-regression network. Joints and associations between joint pairs in our instrument model are located by the detection subnetwork and are subsequently refined through a regression subnetwork. Based on the output from the model, the poses of the instruments are inferred using maximum bipartite graph matching. Our estimation framework is powered by deep learning techniques without any direct kinematic information from a robot. Our framework is tested on single-instrument RMIT data, and also on multi-instrument EndoVis and in vivo data with promising results. In addition, the data set annotations are publicly released along with our code and model.

show abstract

Section: Introductionmentioning

confidence: 99%

Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks

Kurmann

Chang³

et al. 2018

IEEE Trans. Med. Imaging

107

111

View full text Add to dashboard Cite

show abstract

“…Tables III and IV show that the texture descriptors, in their standard form, do not appear to be capable of effectively distinguishing tools and tissue, and due to their significant computation time, they were not included in the feature vector used to train our RF. It is important to note that as shown in [18], descriptors can be used to identify specific points on the articulated tool, but our findings suggest that they are not suitable for whole tool identification. Further work is required to determine whether texture descriptors would be suitable to deliniating the instrument-tissue boundary rather than the tool body itself.…”

Section: A Feature Selection For Classificationmentioning

confidence: 77%

“…They provide an accurate, fast, and potentially parallelizable classification method and offer an easy extension to multiclass data, a useful feature for classifying multiple distinct tool or tissue types [18]. The success of RFs has been due to their 1 http://www.cs.ucl.ac.uk/staff/m.allan/ good generalization ability which increases with the number of trees in the forest combined with the robustness to noise that randomness provides.…”

Section: Appearance Learning With Random Forestsmentioning

confidence: 99%

Toward Detection and Localization of Instruments in Minimally Invasive Surgery

Allan

Ourselin

Thompson

et al. 2013

IEEE Trans. Biomed. Eng.

117

101

View full text Add to dashboard Cite

Methods for detecting and localizing surgical instruments in laparoscopic images are an important element of advanced robotic and computer-assisted interventions. Robotic joint encoders and sensors integrated or mounted on the instrument can provide information about the tool's position, but this often has inaccuracy when transferred to the surgeon's point of view. Vision sensors are currently a promising approach for determining the position of instruments in the coordinate frame of the surgical camera. In this study, we propose a vision algorithm for localizing the instrument's pose in 3-D leaving only rotation in the axis of the tool's shaft as an ambiguity. We propose a probabilistic supervised classification method to detect pixels in laparoscopic images that belong to surgical tools. We then use the classifier output to initialize an energy minimization algorithm for estimating the pose of a prior 3-D model of the instrument within a level set framework. We show that the proposed method is robust against noise using simulated data and we perform quantitative validation of the algorithm compared to ground truth obtained using an optical tracker. Finally, we demonstrate the practical application of the technique on in vivo data from minimally invasive surgery with traditional laparoscopic and robotic instruments.

show abstract

“…This occurs because we only include one feature on the shaft (Pin4), but in the future we will look to include more shaft information. Concurrent with this work we performed a study [21] on the feature detection accuracy, where we obtained an average localization accuracy of 86%, although this varies depending on the feature type. We also note that although some feature types are not always detected, we need only ∼ 3 − 4 on a given frame because of the fusion, and so across the 7 chosen landmarks our experiments show that the percent correct achieved is sufficient for long-term tracking.…”

Section: Resultsmentioning

confidence: 99%

Feature Classification for Tracking Articulated Surgical Tools

Reiter

Allen

Zhao

2012

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Tool tracking is an accepted capability for computer-aided surgical intervention which has numerous applications, both in robotic and manual minimally-invasive procedures. In this paper, we describe a tracking system which learns visual feature descriptors as class-specific landmarks on an articulated tool. The features are localized in 3D using stereo vision and are fused with the robot kinematics to track all of the joints of the dexterous manipulator. Experiments are performed using previously-collected porcine data from a surgical robot.

show abstract

Learning features on robotic surgical tools

Cited by 8 publications

References 18 publications

Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks

Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks

Toward Detection and Localization of Instruments in Minimally Invasive Surgery

Feature Classification for Tracking Articulated Surgical Tools

Contact Info

Product

Resources

About