Contour-based surgical instrument tracking supported by kinematic prediction

Abstract

Abstract-Surgical tool tracking is an important key functionality for many high-level tasks in both robot-assisted and conventional minimally invasive surgery. Though the fields of application are similar in both surgery techniques (i.e. visually servoed instruments, workflow analysis or augmented reality), the kind of available information about the position and orientation of the surgical tool differ. In conventional laparoscopic surgery additional information to the images provides by the endoscopic camera can only be obtained by an external tracking system. In contrast, robotic systems provide angular informations from encoder readings that allow for a sufficient pose estimation and initialization of an image-based tracking algorithm. Our approach utilizes both encoder readings and visual information, in order to stabilize tracking in image space. The image-based tracking is supervised by means of the kinematic information and reinitialized in case of conflicting results. As tracking modality we utilize the Contracting Curve Density (CCD) algorithm that looks for maximal separation of local color statistics along the contour of a model.

Remote Minimally Invasive Surgery – Haptic Feedback and Selective Automation in Medical Robotics

“…Admittedly, an autonomous generation of visually followed paths is desirable and under ongoing work. Also the image processing and tracking part is under improvement and extension (see [29] for first results). In a next step, we will combine the visual servoing with the thread detection of our system [21] in order to handle surgical suture material automatically during knot-tying.…”

A fully-automated multiscale kernel graph cuts based particle localization scheme for temporal focusing two-photon microscopy

“…After precise particle segmentation, a three-volume best-estimate tracking algorithm using kinematic prediction was utilized to tracking the 3D trajectory for segmented particles [8]. Firstly, it predicts the position of particle based on the former two volumes and considers that the moving length and direction of current particle is the same as previous one.…”

Human-computer interfaces for interaction with surgical tools in robotic surgery

“…A main difficulty of the method is the fault-tolerant and reliable detection of the instrument pose. The methods proposed in literature range from instruments, which are labeled with artificial color markers to markerless methods, which e.g., maximize the difference between forand background color alongside a geometrical shape [9], and machine learning techniques that aim to learn and segment the instrument's appearance from the background. A problem shared by most methods is the handling of occlusion (by tissue, body-liquids, or other surgical tools) and changing appearance of the instruments and background, e.g., caused by non-uniform, varying lightning conditions, smoke caused by electro-dissection, and organ movement.…”