Automated Surgical Instrument Detection from Laparoscopic Gastrectomy Video Images Using an Open Source Convolutional Neural Network Platform

Yamazaki, Yuta; Kanaji, Shingo; Matsuda, Takeru; Oshikiri, Taro; Suzuki, Satoshi; Hiasa, Yuta; Otake, Yoshito; Sato, Yoshinobu; Kakeji, Yoshihiro

doi:10.1016/j.jamcollsurg.2020.01.037

Cited by 59 publications

(27 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AI algorithms, particularly those for deep learning, have advanced considerably in medical image-recognition tasks such as radiography 29 – 31 , endoscopy 32 , 33 , and pathological diagnosis 34 , 35 , but their applications to surgery are still being investigated. Many attempts have aimed to recognize surgical instruments 36 or operative workflows such as cholecystectomy 37 – 39 , colectomy 40 , and sleeve gastrectomy 41 . Madani et al reported promising results for the identification of safe zones for dissection during laparoscopic cholecystectomy (defined as the area located within the hepatocystic triangle), with high sensitivity and F1/Dice scores of 0.69 and 0.70, respectively 37 .…”

Section: Discussionmentioning

confidence: 99%

Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy

Kumazu

Kobayashi²,

Kitamura³

et al. 2021

Sci Rep

View full text Add to dashboard Cite

The prediction of anatomical structures within the surgical field by artificial intelligence (AI) is expected to support surgeons’ experience and cognitive skills. We aimed to develop a deep-learning model to automatically segment loose connective tissue fibers (LCTFs) that define a safe dissection plane. The annotation was performed on video frames capturing a robot-assisted gastrectomy performed by trained surgeons. A deep-learning model based on U-net was developed to output segmentation results. Twenty randomly sampled frames were provided to evaluate model performance by comparing Recall and F1/Dice scores with a ground truth and with a two-item questionnaire on sensitivity and misrecognition that was completed by 20 surgeons. The model produced high Recall scores (mean 0.606, maximum 0.861). Mean F1/Dice scores reached 0.549 (range 0.335–0.691), showing acceptable spatial overlap of the objects. Surgeon evaluators gave a mean sensitivity score of 3.52 (with 88.0% assigning the highest score of 4; range 2.45–3.95). The mean misrecognition score was a low 0.14 (range 0–0.7), indicating very few acknowledged over-detection failures. Thus, AI can be trained to predict fine, difficult-to-discern anatomical structures at a level convincing to expert surgeons. This technology may help reduce adverse events by determining safe dissection planes.

show abstract

Section: Discussionmentioning

confidence: 99%

Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy

Kumazu

Kobayashi²,

Kitamura³

et al. 2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Phases and instrument usage information could be used to synchronize videos of the same surgical procedure, allowing for smart indexing and efficient querying of large databases of surgical videos. As recently shown by a Japanese group analyzing gastrectomies, 52 surgical instrument usage patterns could be plotted to efficiently screen for cases and scenes likely to show unexpected events so as to prioritize their VBA. The Surgical AI and Innovation Laboratory at Massachusetts General Hospital have described the concept of the “surgical fingerprint” wherein phase recognition algorithms can assess the video of interest's workflow against that of a pre‐existing database to determine whether the video is following an expected operative course.…”

Section: Introductionmentioning

confidence: 99%

Surgical data science and artificial intelligence for surgical education

Ward

Mascagni

Madani

et al. 2021

Journal of Surgical Oncology

View full text Add to dashboard Cite

Surgical data science (SDS) aims to improve the quality of interventional healthcare and its value through the capture, organization, analysis, and modeling of procedural data. As data capture has increased and artificial intelligence (AI) has advanced, SDS can help to unlock augmented and automated coaching, feedback, assessment, and decision support in surgery. We review major concepts in SDS and AI as applied to surgical education and surgical oncology.

show abstract

“…Two surgeons reviewed recorded videos of surgeons being trained using the BABA training model and of surgeons performing thyroid surgery on patients with thyroid cancer [ 48 , 49 ]. Parts of items and related motion metrics in OSATS and GEARS were scored [ 31 , 50 ].…”

Section: Methodsmentioning

confidence: 99%

Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations

Lee

Kwon

et al. 2020

JCM

View full text Add to dashboard Cite

As the number of robotic surgery procedures has increased, so has the importance of evaluating surgical skills in these techniques. It is difficult, however, to automatically and quantitatively evaluate surgical skills during robotic surgery, as these skills are primarily associated with the movement of surgical instruments. This study proposes a deep learning-based surgical instrument tracking algorithm to evaluate surgeons’ skills in performing procedures by robotic surgery. This method overcame two main drawbacks: occlusion and maintenance of the identity of the surgical instruments. In addition, surgical skill prediction models were developed using motion metrics calculated from the motion of the instruments. The tracking method was applied to 54 video segments and evaluated by root mean squared error (RMSE), area under the curve (AUC), and Pearson correlation analysis. The RMSE was 3.52 mm, the AUC of 1 mm, 2 mm, and 5 mm were 0.7, 0.78, and 0.86, respectively, and Pearson’s correlation coefficients were 0.9 on the x-axis and 0.87 on the y-axis. The surgical skill prediction models showed an accuracy of 83% with Objective Structured Assessment of Technical Skill (OSATS) and Global Evaluative Assessment of Robotic Surgery (GEARS). The proposed method was able to track instruments during robotic surgery, suggesting that the current method of surgical skill assessment by surgeons can be replaced by the proposed automatic and quantitative evaluation method.

show abstract

Automated Surgical Instrument Detection from Laparoscopic Gastrectomy Video Images Using an Open Source Convolutional Neural Network Platform

Cited by 59 publications

References 11 publications

Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy

Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy

Surgical data science and artificial intelligence for surgical education

Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations

Contact Info

Product

Resources

About