An intraoperative artificial intelligence system identifying anatomical landmarks for laparoscopic cholecystectomy: a prospective clinical feasibility trial (J-SUMMIT-C-01)

Nakanuma, Hiroaki; Endo, Yuichi; Fujinaga, Atsuro; Kawamura, Masahiro; Kawasaki, Takahide; Masuda, Takashi; Hirashita, Teijiro; Etoh, Tsuyoshi; Shinozuka, Ken’ichi; Matsunobu, Yusuke; Kamiyama, Tetsuo; Ishikake, Makoto; Ebe, Kohei; Tokuyasu, Tatsushi; Ito, Masafumi

doi:10.1007/s00464-022-09678-w

Cited by 13 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nakanuma et al recently published a feasibility trial (J-SUMMIT-C-01) for a YOLOv3-based object detection framework to be used for intraoperative guidance in laparoscopic cholecystectomy (LC). Although they used the YOLOv3 framework, they were able to demonstrate an objective usefulness of an AI-powered surgical guidance platform ( 11 , 12 ). In addition, Liu et al provided supporting evidence that their YOLOv3 based framework identified anatomical structures within LC more accurately than their trainees and senior surgeons ( 13 ).…”

Section: Discussionmentioning

confidence: 99%

“…Instead, metrics such as the F1-score, precision, and recall provide a more accurate evaluation ( 25 ). Nakanuma et al used the DICE coefficient as a metric for accuracy; however, we have incorporated this into our loss function ( 12 ).…”

Section: Discussionmentioning

confidence: 99%

“…Therefore, to minimise bias and subjectivity ( 7 , 26 ), we plan to involve multiple experts in annotating the surgical images and assess the inter-rater reliability. Additionally, it is important to note that spinal dAVF surgery is a microscopic procedure that does not currently allow our platform to provide superimposed guidance during surgery, as seen in many laparoscopic studies ( 7 , 11 , 12 ). To address this limitation, our research group is currently exploring an upgraded architecture that can process sparsely labelled data from multiple patients who have undergone endoscopic microvascular decompression (MVD) surgery ( 27 ).…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Developing the surgeon-machine interface: using a novel instance-segmentation framework for intraoperative landmark labelling

Park,

Doiphode,

Zhang

et al. 2023

Front. Surg.

View full text Add to dashboard Cite

IntroductionThe utilisation of artificial intelligence (AI) augments intraoperative safety, surgical training, and patient outcomes. We introduce the term Surgeon-Machine Interface (SMI) to describe this innovative intersection between surgeons and machine inference. A custom deep computer vision (CV) architecture within a sparse labelling paradigm was developed, specifically tailored to conceptualise the SMI. This platform demonstrates the ability to perform instance segmentation on anatomical landmarks and tools from a single open spinal dural arteriovenous fistula (dAVF) surgery video dataset.MethodsOur custom deep convolutional neural network was based on SOLOv2 architecture for precise, instance-level segmentation of surgical video data. Test video consisted of 8520 frames, with sparse labelling of only 133 frames annotated for training. Accuracy and inference time, assessed using F1-score and mean Average Precision (mAP), were compared against current state-of-the-art architectures on a separate test set of 85 additionally annotated frames.ResultsOur SMI demonstrated superior accuracy and computing speed compared to these frameworks. The F1-score and mAP achieved by our platform were 17% and 15.2% respectively, surpassing MaskRCNN (15.2%, 13.9%), YOLOv3 (5.4%, 11.9%), and SOLOv2 (3.1%, 10.4%). Considering detections that exceeded the Intersection over Union threshold of 50%, our platform achieved an impressive F1-score of 44.2% and mAP of 46.3%, outperforming MaskRCNN (41.3%, 43.5%), YOLOv3 (15%, 34.1%), and SOLOv2 (9%, 32.3%). Our platform demonstrated the fastest inference time (88ms), compared to MaskRCNN (90ms), SOLOV2 (100ms), and YOLOv3 (106ms). Finally, the minimal amount of training set demonstrated a good generalisation performance –our architecture successfully identified objects in a frame that were not included in the training or validation frames, indicating its ability to handle out-of-domain scenarios.DiscussionWe present our development of an innovative intraoperative SMI to demonstrate the future promise of advanced CV in the surgical domain. Through successful implementation in a microscopic dAVF surgery, our framework demonstrates superior performance over current state-of-the-art segmentation architectures in intraoperative landmark guidance with high sample efficiency, representing the most advanced AI-enabled surgical inference platform to date. Our future goals include transfer learning paradigms for scaling to additional surgery types, addressing clinical and technical limitations for performing real-time decoding, and ultimate enablement of a real-time neurosurgical guidance platform.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Developing the surgeon-machine interface: using a novel instance-segmentation framework for intraoperative landmark labelling

Park,

Doiphode,

Zhang

et al. 2023

Front. Surg.

View full text Add to dashboard Cite

show abstract

“…Nevertheless, there is a lack of an overarching AI framework that can learn, train, and segment or label any anatomical structures in a human body. Some intraoperative AI models have been applying deep learning architectures to predict objects that were previously unseen by the AI ( 36 ). Albeit its infancy, this seems to be the future in AI-powered segmentation and anatomical labelling.…”

Section: Discussionmentioning

confidence: 99%

The role of artificial intelligence in surgical simulation

Park

Tiefenbach

Demetriades

2022

Front. Med. Technol.

View full text Add to dashboard Cite

Artificial Intelligence (AI) plays an integral role in enhancing the quality of surgical simulation, which is increasingly becoming a popular tool for enriching the training experience of a surgeon. This spans the spectrum from facilitating preoperative planning, to intraoperative visualisation and guidance, ultimately with the aim of improving patient safety. Although arguably still in its early stages of widespread clinical application, AI technology enables personal evaluation and provides personalised feedback in surgical training simulations. Several forms of surgical visualisation technologies currently in use for anatomical education and presurgical assessment rely on different AI algorithms. However, while it is promising to see clinical examples and technological reports attesting to the efficacy of AI-supported surgical simulators, barriers to wide-spread commercialisation of such devices and software remain complex and multifactorial. High implementation and production costs, scarcity of reports evidencing the superiority of such technology, and intrinsic technological limitations remain at the forefront. As AI technology is key to driving the future of surgical simulation, this paper will review the literature delineating its current state, challenges, and prospects. In addition, a consolidated list of FDA/CE approved AI-powered medical devices for surgical simulation is presented, in order to shed light on the existing gap between academic achievements and the universal commercialisation of AI-enabled simulators. We call for further clinical assessment of AI-supported surgical simulators to support novel regulatory body approved devices and usher surgery into a new era of surgical education.

show abstract

“…The emerging field of surgical phase recognition using deep learning is evidenced in several studies [ [14] , [15] , [16] ], as is the identification of surgical instruments [ [17] , [18] , [19] ]. Moreover, numerous researchers have leveraged deep learning to detect anatomical landmarks during surgical procedures [ [20] , [21] , [22] ]. Tokuyasu et al [ 23 ] devised a real-time object detection model based on YOLOv3 for identifying four landmarks, which displays the bounding box of the cystic duct, common bile duct, lower edge of the left medial segment, and Rouviere's sulcus on monitors during LC.…”

Section: Introductionmentioning

confidence: 99%

Development of deep learning framework for anatomical landmark detection and guided dissection line during laparoscopic cholecystectomy

Smithmaitrie,

Khaonualsri,

Sae-Lim

et al. 2024

Heliyon

View full text Add to dashboard Cite

An intraoperative artificial intelligence system identifying anatomical landmarks for laparoscopic cholecystectomy: a prospective clinical feasibility trial (J-SUMMIT-C-01)

Cited by 13 publications

References 27 publications

Developing the surgeon-machine interface: using a novel instance-segmentation framework for intraoperative landmark labelling

Developing the surgeon-machine interface: using a novel instance-segmentation framework for intraoperative landmark labelling

The role of artificial intelligence in surgical simulation

Development of deep learning framework for anatomical landmark detection and guided dissection line during laparoscopic cholecystectomy

Contact Info

Product

Resources

About