Towards real-time multiple surgical tool tracking

Abstract

Surgical tool tracking is an essential building block for computer assisted interventions (CAI) and applications like video summarization, workflow analysis and surgical navigation. Vision-based instrument tracking in laparoscopic surgical data faces significant challenges such as fast instrument motion, multiple simultaneous instruments, and re-initialization due to out-of-view conditions or instrument occlusions. In this paper, we propose a real-time multiple object tracking framework for whole laparoscopic tools, which extends an existing single object tracker. We introduce a geometric object descriptor, which helps with overlapping bounding box disambiguation, fast motion and optimal assignment between existing trajectories and new hypotheses. We achieve 99.51% and 75.64% average accuracy on ex-vivo robotic data and in-vivo laparoscopic sequences respectively from the Endovis'15 Instrument Tracking Dataset. The proposed geometric descriptor increased the performance on laparoscopic data by 32%, significantly reducing identity switches, false negatives and false positives. Overall, the proposed pipeline can successfully recover trajectories over long-sequences and it runs in real-time at approximately 25-29 fps.

Ethical implications of AI in robotic surgical training: A Delphi consensus statement

“…The deployment of AI in surgical training is still early in its development. Although significant advances are being made, these advances are focused on narrow applications of the technology to specific aspects of performance such as surgical instrument tracking [29]. One of the largest gaps in current knowledge is understanding the scope of the domain of data and the variability that it may contain.…”

Real-Time Biomedical Imaging for Surgical Guidance: A Review of Challenges and Solutions

“…Despite the theoretical advantages, the practical application of quantum computing in biomedical imaging is in its infancy. Current quantum computers, often referred to as Noisy Intermediate-Scale Quantum (NISQ) devices, are limited by issues such as quantum decoherence and error rates [38]. However, continuous progress in quantum error correction and the development of faulttolerant quantum computers are rapidly bridging the gap between theory and practice.…”

Tracking and Estimation of Surgical Instrument Position and Angle in Surgical Robot Using Vision System

“…Thereafter, a faster R-CNNbased modified Anchoring Network was prepared to identify the instrument during the key-hole operation [5]. A geometric object descriptor-based multiple tool tracing architecture was introduced for limited surgical tool data sets [6].…”