3-D Pose Estimation of Articulated Instruments in Robotic Minimally Invasive Surgery

Allan, Maximilian; Ourselin, Sébastien; Hawkes, David J.; Kelly, John D.; Stoyanov, Danail

doi:10.1109/tmi.2018.2794439

Cited by 72 publications

(38 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In terms of applications, it should be noted that most solutions were developed to monitor endoscopic videos for minimally invasive surgeries, with or without robotic assistance (Sarikaya et al, 2017;Ross et al, 2018;Wesierski and Jezierska, 2018;Du et al, 2018;Allan et al, 2018). Other imaging modalities include:…”

Section: Clinical Applicationsmentioning

confidence: 99%

“…For flexible instruments, the goal is also to detect the tool centerline (Chang et al, 2016). Tool detection generally is an intermediate step for tool tracking, the process of monitoring tool location over time (Du et al, 2016;Rieke et al, 2016a;Lee et al, 2017b;Zhao et al, 2017;Czajkowska et al, 2018;Ryu et al, 2018;Keller et al, 2018), and pose estimation, the process of inferring a 2-D pose (Rieke et al, 2016b;Kurmann et al, 2017;Alsheakhali et al, 2016b;Du et al, 2018;Wesierski and Jezierska, 2018) or a 3-D pose (Allan et al, 2018;Gessert et al, 2018) based on the location of tool elements. Tasks associated with tool detection also include velocity estimation (Marban et al, 2017) and instrument state recognition (Sahu et al, 2016a).…”

Section: Computer Vision Tasksmentioning

confidence: 99%

See 1 more Smart Citation

CATARACTS: Challenge on automatic tool annotation for cataRACT surgery

Hajj¹,

Lamard²,

Conze³

et al. 2019

Medical Image Analysis

Self Cite

View full text Add to dashboard Cite

Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design efficient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparoscopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected, the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the differential analysis of these solutions are discussed. We expect that they will guide the design of efficient surgery monitoring tools in the near future.

show abstract

Section: Clinical Applicationsmentioning

confidence: 99%

Section: Computer Vision Tasksmentioning

confidence: 99%

CATARACTS: Challenge on automatic tool annotation for cataRACT surgery

Hajj¹,

Lamard²,

Conze³

et al. 2019

Medical Image Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, many challenges in algorithm robustness hampering the translation of CAI methods relying on computer vision to the clinical practice. These include classification and segmentation of organs in the camera field of view (FoV) [3], definition of virtual-fixture algorithms to impose a safe distance between surgical tools and sensitive tissues [4], and surgical instrument detection, segmentation and articulated pose estimation [5], [6].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Based Robotic Tool Detection and Articulation Estimation With Spatio-Temporal Layers

Colleoni

Moccia

et al. 2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Surgical-tool detection from laparoscopic images is an important but challenging task in computer-assisted minimally invasive surgery. Illumination levels, variations in background and the different number of tools in the field of view, all pose difficulties to algorithm and model training. Yet, such challenges could be potentially tackled by exploiting the temporal information in laparoscopic videos to avoid per frame handling of the problem. In this paper, we propose a novel encoderdecoder architecture for surgical instrument detection and articulation joint detection that uses 3D convolutional layers to exploit spatio-temporal features from laparoscopic videos. When tested on benchmark and custom-built datasets, a median Dice similarity coefficient of 85.1% with an interquartile range of 4.6% highlights performance better than the state of the art based on single-frame processing. Alongside novelty of the network architecture, the idea for inclusion of temporal information appears to be particularly useful when processing images with unseen backgrounds during the training phase, which indicates that spatio-temporal features for joint detection help to generalize the solution.

show abstract

“…Therefore, robotic tool detection, segmentation, tracking and pose estimation are bound to become core technologies in a surgical work-flow in improving planning and understanding during the operation. In the context of delicate surgical procedures, such as urology, 3 it is paramount to provide the clinical operator with accurate real-time information about tool-tissue interactions, 4 3D position and orientation of the instruments, 5 etc., to increase the context-awareness of the operator whilst performing robotic intervention and helping to avoid human errors.…”

Section: Introductionmentioning

confidence: 99%

StreoScenNet: surgical stereo robotic scene segmentation

Mohammed

Yayilgan

Farup

et al. 2019

Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling

View full text Add to dashboard Cite

Surgical robot technology has revolutionized surgery toward a safer laparoscopic surgery and ideally been suited for surgeries requiring minimal invasiveness. Sematic segmentation from robot-assisted surgery videos is an essential task in many computer-assisted robotic surgical systems. Some of the applications include instrument detection, tracking and pose estimation. Usually, the left and right frames from the stereoscopic surgical instrument are used for semantic segmentation independently from each other. However, this approach is prone to poor segmentation since the stereo frames are not integrated for accurate estimation of the surgical scene. To cope with this problem, we proposed a multi encoder and single decoder convolutional neural network named StreoScenNet which exploits the left and right frames of the stereoscopic surgical system. The proposed architecture consists of multiple ResNet encoder blocks and a stacked convolutional decoder network connected with a novel sum-skip connection. The input to the network is a set of left and right frames and the output is a mask of the segmented regions for the left frame. It is trained end-to-end and the segmentation is achieved without the need of any pre-or post-processing. We compare the proposed architectures against state-of-the-art fully convolutional networks. We validate our methods using existing benchmark datasets that includes robotic instruments as well as anatomical objects and non-robotic surgical instruments. Compared with the previous instrument segmentation methods, our approach achieves a significant improved Dice similarity coefficient.

show abstract

3-D Pose Estimation of Articulated Instruments in Robotic Minimally Invasive Surgery

Cited by 72 publications

References 42 publications

CATARACTS: Challenge on automatic tool annotation for cataRACT surgery

CATARACTS: Challenge on automatic tool annotation for cataRACT surgery

Deep Learning Based Robotic Tool Detection and Articulation Estimation With Spatio-Temporal Layers

StreoScenNet: surgical stereo robotic scene segmentation

Contact Info

Product

Resources

About