Surgical Instrument Detection for Laparoscopic Surgery using Deep Learning

Boonkong, Apiwat; Hormdee, Daranee; Sonsilphong, Suphachoke; Khampitak, Kovit

doi:10.1109/ecti-con54298.2022.9795561

Cited by 8 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One part of the research groups is concerned with recognising instruments used in minimally invasive procedures such as laparoscopy. In their work, Boonkong, A. et al 4 train a neural network to recognise surgical instruments during a laproscopic procedure. The other area of research in this field is the recognition of surgical instruments used in open surgery.…”

Section: Surgical Instrument Detectionmentioning

confidence: 99%

Medical instrument detection with synthetically generated data

Wiese,

Hinz,

Reithmeier

2024

Medical Imaging 2024: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

The persistent need for more qualified personnel in operating theatres exacerbates the remaining staff's workload. This increased burden can result in substantial complications during surgical procedures. To address this issue, this research project works on a comprehensive operating theatre system. The system offers real-time monitoring of all surgical instruments in the operating theatre, aiming to alleviate the problem. The foundation of this endeavour involves a neural network trained to classify and identify eight distinct instruments belonging to four distinct surgical instrument groups. A novel aspect of this study lies in the approach taken to select and generate the training and validation data sets. The data sets used in this study consist of synthetically generated image data rather than real image data. Additionally, three virtual scenes were designed to serve as the background for a generation algorithm. This algorithm randomly positions the instruments within these scenes, producing annotated rendered RGB images of the generated scenes. To assess the efficacy of this approach, a separate real data set was also created for testing the neural network. Surprisingly, it was discovered that neural networks trained solely on synthetic data performed well when applied to real data. This research paper shows that it is possible to train neural networks with purely synthetically generated data and use them to recognise surgical instruments in real images.

show abstract

Section: Surgical Instrument Detectionmentioning

confidence: 99%

Medical instrument detection with synthetically generated data

Wiese,

Hinz,

Reithmeier

2024

Medical Imaging 2024: Imaging Informatics for Healthcare, Research, and Applications

View full text Add to dashboard Cite

show abstract

“…Surgical tools are the most important actuators in surgery because they are responsible for performing interventions; however, keeping track of surgical instruments requires realtime knowledge of the pose and the movement of the tool. Literature suggests numerous tool localisation techniques embracing electromagnetic tracking [63], kinematic [64], optical tracing [65], and image-guided detection [66] among others [67]. Unlike other approaches, image driven surgical instrument localisation offers attractive benefits including the knowledge of pose and motion, and does not require instrument design modification [47].…”

Section: A Surgical Toolsmentioning

confidence: 99%

“…The CNN is the most applied DL method followed by Long Short Term Memory (LSTM), Recurrent Neural Network (RNN) and autoencoder architectures (Figure 5). The CNN model and the variants, with few modifications in the underlying architecture in some cases, yielded better performance in [29], [67]- [70], [72]- [75], [77], [79], [80], [82]- [85], [87]- [92], [94], [97], [98], [100], [101], whereas autoencoders, RNN, LSTM, and Generative Adversarial Network (GAN) formed another notable synergy [39], [71], [76], [78], [81], [86], [93], [95].…”

Section: ) Tool Detectionmentioning

confidence: 99%

“…The reason behind CNN being the most applied architecture lies in the ability of multiple tool detection and localisation which traditional ML models have not been sufficiently successful at [67]. Among the tool detection, the articles employing public datasets revealed an accuracy range of 89-100%, whereas, the precision and Dice values vary greatly.…”

Section: ) Tool Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

et al. 2022

View full text Add to dashboard Cite

The recent advancements in the surging field of Deep Learning (DL) have revolutionized every sphere of life, and the healthcare domain is no exception. The enormous success of DL models, particularly with image data, has led to the development of image-guided Robot Assisted Surgery (RAS) systems. By and large, the number of studies concerning image-driven computer assisted surgical systems using DL has increased exponentially. Additionally, the contemporary availability of surgical datasets has also boosted the DL applications in RAS. Inspired by the latest trends and contributions in surgery, this literature survey presents a summarized analysis of recent innovations of DL in image-guided RAS systems. After a thorough review, a sum of 184 articles are selected and grouped into four categories, based on the literature and the relevancy of the task in the articles, comprising 1) Surgical Tools, 2) Surgical Processes, 3) Surgical Surveillance, and 4) Surgical Performance. The survey also discusses publicly available surgical datasets and highlights the basics of the DL models. Furthermore, the legal, ethical, and technological challenges together with the intuitive predictions and recommendations related to the autonomous RAS systems are also presented. The study reveals that Convolutional Neural Network (CNN) is most widely adopted architecture, whereas, JIGSAWS is most employed dataset in RAS. The study suggests fusing kinematic data along with image data, which produces better accuracy and precision, particularly in gesture and trajectory segmentation tasks. Additionally, CNN and Long Short Term Memory (LSTM) networks have shown remarkable performance, however, authors recommend employing these gigantic architectures only when simpler models have failed to produce satisfactory results. The simpler models, despite their limitations, are time and cost effective and yield considerable outcomes even on the smaller datasets.

show abstract

Extra-abdominal trocar and instrument detection for enhanced surgical workflow understanding

Jurosch,

Wagner,

Jell

et al. 2024

Int J CARS

View full text Add to dashboard Cite

Purpose Video-based intra-abdominal instrument tracking for laparoscopic surgeries is a common research area. However, the tracking can only be done with instruments that are actually visible in the laparoscopic image. By using extra-abdominal cameras to detect trocars and classify their occupancy state, additional information about the instrument location, whether an instrument is still in the abdomen or not, can be obtained. This can enhance laparoscopic workflow understanding and enrich already existing intra-abdominal solutions. Methods A data set of four laparoscopic surgeries recorded with two time-synchronized extra-abdominal 2D cameras was generated. The preprocessed and annotated data were used to train a deep learning-based network architecture consisting of a trocar detection, a centroid tracker and a temporal model to provide the occupancy state of all trocars during the surgery. Results The trocar detection model achieves an F1 score of $$95.06\pm 0.88\%$$ 95.06 ± 0.88 % . The prediction of the occupancy state yields an F1 score of $$89.29\pm 5.29\%$$ 89.29 ± 5.29 % , providing a first step towards enhanced surgical workflow understanding. Conclusion The current method shows promising results for the extra-abdominal tracking of trocars and their occupancy state. Future advancements include the enlargement of the data set and incorporation of intra-abdominal imaging to facilitate accurate assignment of instruments to trocars.

show abstract

Surgical Instrument Detection for Laparoscopic Surgery using Deep Learning

Cited by 8 publications

References 18 publications

Medical instrument detection with synthetically generated data

Medical instrument detection with synthetically generated data

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

Extra-abdominal trocar and instrument detection for enhanced surgical workflow understanding

Contact Info

Product

Resources

About