Automatic Detection of Instruments in Laparoscopic Images: A First Step Towards High-level Command of Robotic Endoscopic Holders

Abstract. In the context of retinal microsurgery, visual tracking of instruments is a key component of robotics assistance. The difficulty of the task and major reason why most existing strategies fail on in-vivo image sequences lies in the fact that complex and severe changes in instrument appearance are challenging to model. This paper introduces a novel approach, that is both data-driven and complementary to existing tracking techniques. In particular, we show how to learn and integrate an accurate detector with a simple gradient-based tracker within a robust pipeline which runs at framerate. In addition, we present a fully annotated dataset of retinal instruments in in-vivo surgeries, which we use to quantitatively validate our approach. We also demonstrate an application of our method in a laparascopy image sequence.

show abstract

“…1. Note that a similar observation can also be made regarding tool tracking techniques for laparoscopic surgery [7].…”

Section: Introductionmentioning

confidence: 61%

“…5(right)), 2) occluded instruments are not found given that there is no geometrical model to help with such situations. A possible alternative to overcome this may be to integrate our approach with more elaborate prior instrument knowledge (as in [3,4,7]). …”

Section: Laparoscopy Sequencementioning

confidence: 99%

Data-Driven Visual Tracking in Retinal Microsurgery

Sznitman

Ali

Richa

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Some approaches address this problem by using artificial markers on the instrument (48,49). However, some of these approaches have some complications including the ambiguous image structures (52) and occlusions by blood, organs or other instruments, or smoke caused by electrodissection (37,47,49,54). Both real-time image processing and the third dimension of the instrument's position are critical factors in these instrument tracking methods.…”

Section: Discussionmentioning

confidence: 99%

“…In order to avoid them, some researchers analyse the 3D position of the instrument tip based on its projection in the image plane (Figure 3). Additionally, Voros et al (52) restrict the search of the laparoscopic instrument by means of computing the projection of the instrument's insertion point into the abdominal cavity. Other approaches use a Bayesian classifier (53) or a Cascade of Boosted Classifiers (54) previously trained to identify the laparoscopic instrument (Figure 4).…”

Section: Intra-corporeal Systemsmentioning

confidence: 99%

Systems and technologies for objective evaluation of technical skills in laparoscopic surgery

Sánchez‐Margallo

García

et al. 2013

Minimally Invasive Therapy & Allied Technologies

View full text Add to dashboard Cite

Minimally invasive surgery is a highly demanding surgical approach regarding technical requirements for the surgeon, who must be trained in order to perform a safe surgical intervention. Traditional surgical education in minimally invasive surgery is commonly based on subjective criteria to quantify and evaluate surgical abilities, which could be potentially unsafe for the patient. Authors, surgeons and associations are increasingly demanding the development of more objective assessment tools that can accredit surgeons as technically competent. This paper describes the state of the art in objective assessment methods of surgical skills. It gives an overview on assessment systems based on structured checklists and rating scales, surgical simulators, and instrument motion analysis. As a future work, an objective and automatic assessment method of surgical skills should be standardized as a means towards proficiency-based curricula for training in laparoscopic surgery and its certification.

show abstract

“…Therefore, the fulcrum is estimated with a series of images in order to project an approximated instrument direction and shape into the image. Voros et al [15] also reduces the search space by considering the insertion point of the instrument. At the beginning of the procedure, the fulcrum has to be visible in the image and is marked with a "vocal mouse".…”

Section: Introductionmentioning

confidence: 99%

Contour-based surgical instrument tracking supported by kinematic prediction

Staub¹,

Lenz²,

Panin³

et al. 2010

2010 3rd IEEE RAS &Amp; EMBS International Conference on Biomedical Robotics and Biomechatronics

View full text Add to dashboard Cite

Abstract-Surgical tool tracking is an important key functionality for many high-level tasks in both robot-assisted and conventional minimally invasive surgery. Though the fields of application are similar in both surgery techniques (i.e. visually servoed instruments, workflow analysis or augmented reality), the kind of available information about the position and orientation of the surgical tool differ. In conventional laparoscopic surgery additional information to the images provides by the endoscopic camera can only be obtained by an external tracking system. In contrast, robotic systems provide angular informations from encoder readings that allow for a sufficient pose estimation and initialization of an image-based tracking algorithm. Our approach utilizes both encoder readings and visual information, in order to stabilize tracking in image space. The image-based tracking is supervised by means of the kinematic information and reinitialized in case of conflicting results. As tracking modality we utilize the Contracting Curve Density (CCD) algorithm that looks for maximal separation of local color statistics along the contour of a model.

show abstract

Automatic Detection of Instruments in Laparoscopic Images: A First Step Towards High-level Command of Robotic Endoscopic Holders

Cited by 87 publications

References 20 publications

Data-Driven Visual Tracking in Retinal Microsurgery

Data-Driven Visual Tracking in Retinal Microsurgery

Systems and technologies for objective evaluation of technical skills in laparoscopic surgery

Contour-based surgical instrument tracking supported by kinematic prediction

Contact Info

Product

Resources

About