Simultaneous Recognition and Pose Estimation of Instruments in Minimally Invasive Surgery

Kurmann, Thomas; Neila, Pablo Márquez; Du, Xiaofei; Fua, Pascal; Stoyanov, Danail; Wolf, Sebastián; Sznitman, Raphael

doi:10.1007/978-3-319-66185-8_57

Cited by 69 publications

(48 citation statements)

References 11 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the literature, surgical tool localization has traditionally been approached with fully supervised methods [1], with the most recent localization and segmentation methods relying on deep learning [4,8,10,11,13]. However, training fully supervised approaches require the data to be fully annotated with spatial information, which is tedious and expensive.…”

Section: Introductionmentioning

confidence: 99%

Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Vardazaryan

Mutter

Marescaux

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Surgical tool localization is an essential task for the automatic analysis of endoscopic videos. In the literature, existing methods for tool localization, tracking and segmentation require training data that is fully annotated, thereby limiting the size of the datasets that can be used and the generalization of the approaches. In this work, we propose to circumvent the lack of annotated data with weak supervision. We propose a deep architecture, trained solely on image level annotations, that can be used for both tool presence detection and localization in surgical videos. Our architecture relies on a fully convolutional neural network, trained end-to-end, enabling us to localize surgical tools without explicit spatial annotations. We demonstrate the benefits of our approach on a large public dataset, Cholec80, which is fully annotated with binary tool presence information and of which 5 videos have been fully annotated with bounding boxes and tool centers for the evaluation.

show abstract

Section: Introductionmentioning

confidence: 99%

Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Vardazaryan

Mutter

Marescaux

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Surgical-tool detection in particular has been investigated in recent literature for different surgical fields, such as retinal microsurgery [7] and abdominal MIS [8]. Information provided by algorithms can be used to provide analytical reports, as well as, as a component within CAI frameworks.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Based Robotic Tool Detection and Articulation Estimation With Spatio-Temporal Layers

Colleoni

Moccia

et al. 2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Surgical-tool detection from laparoscopic images is an important but challenging task in computer-assisted minimally invasive surgery. Illumination levels, variations in background and the different number of tools in the field of view, all pose difficulties to algorithm and model training. Yet, such challenges could be potentially tackled by exploiting the temporal information in laparoscopic videos to avoid per frame handling of the problem. In this paper, we propose a novel encoderdecoder architecture for surgical instrument detection and articulation joint detection that uses 3D convolutional layers to exploit spatio-temporal features from laparoscopic videos. When tested on benchmark and custom-built datasets, a median Dice similarity coefficient of 85.1% with an interquartile range of 4.6% highlights performance better than the state of the art based on single-frame processing. Alongside novelty of the network architecture, the idea for inclusion of temporal information appears to be particularly useful when processing images with unseen backgrounds during the training phase, which indicates that spatio-temporal features for joint detection help to generalize the solution.

show abstract

“…Powerful deep learning algorithms, which can learn hierarchical features through their structure and are widely used in the field of computer vision [7,8], have become the mainstream methods in instrument tracking or detection. The modified and extended U-net [9] architecture was proposed by Kurmann et al [10] for 2D vision-based recognition and pose estimation of multiple instruments, while the fully convolutional network [11] (FCN) was employed for tracking [12]. Due to the interdependence of location and segmentation of the surgical instrument, Laina et al [13] proposed to use CSL model to perform instrument segmentation and pose estimation simultaneously.…”

Section: Introductionmentioning

confidence: 99%

Convolutional neural network-based surgical instrument detection

Cai¹,

Zhao²

2020

THC

View full text Add to dashboard Cite

BACKGROUND: Minimally invasive surgery (MIS), unlike open surgery in which surgeons can perform surgery directly, is performed using miniaturized instruments with indirect but careful observation of the surgical site. OBJECTIVE: Instrument detection is a crucial requirement in conventional and robot-assisted MIS, which can also be very useful during surgical training. In this paper, we propose a novel framework of using two three-layer convolutional neural networks (CNNs) in a series to detect surgical instrument in in-vivo video frames. METHODS: The two convolutional neural networks proposed in this paper have different tasks. (i) The former CNN is trained to detect the edges points of the instrument shaft directly from images patches. (ii) The latter is trained to locate the instrument tip also from images patches after the former detection finishes. RESULTS: We validated our method on the publicly available EndoVisSub dataset and a standard dataset, and it detected tools with an accuracy of 91.2% and 75% respectively. CONCLUSION: Our two-step detection method achieves better performance than other existing approaches in terms of detection accuracy.

show abstract

Simultaneous Recognition and Pose Estimation of Instruments in Minimally Invasive Surgery

Cited by 69 publications

References 11 publications

Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Deep Learning Based Robotic Tool Detection and Articulation Estimation With Spatio-Temporal Layers

Convolutional neural network-based surgical instrument detection

Contact Info

Product

Resources

About