Deep Learning-based Biological Anatomical Landmark Detection in Colonoscopy Videos

Abstract

Colonoscopy is a standard imaging tool for visualizing the entire gastrointestinal (GI) tract of patients to capture lesion areas. However, it takes the clinicians excessive time to review a large number of images extracted from colonoscopy videos. Thus, automatic detection of biological anatomical landmarks within the colon is highly demanded, which can help reduce the burden of clinicians by providing guidance information for the locations of lesion areas. In this article, we propose a novel deep learning-based approach to detect biological anatomical landmarks in colonoscopy videos. First, raw colonoscopy video sequences are pre-processed to reject interference frames. Second, a ResNet-101 based network is used to detect three biological anatomical landmarks separately to obtain the intermediate detection results. Third, to achieve more reliable localization of the landmark periods within the whole video period, we propose to post-process the intermediate detection results by identifying the incorrectly predicted frames based on their temporal distribution and reassigning them back to the correct class. Finally, the average detection accuracy reaches 99.75%. Meanwhile, the average IoU of 0.91 shows a high degree of similarity between our predicted landmark periods and ground truth. The experimental results demonstrate that our proposed model is capable of accurately detecting and localizing biological anatomical landmarks from colonoscopy videos.

AI-Guided Anatomical Landmark and Abnormality Detection for Autonomous Endoscopy Examination

“…These methods are suitable for the diagnosis of a specific condition (Hossain et al, 2023;Ozawa et al, 2020;Bour et al, 2019;Tomar et al, 2021;Misawa et al, 2021;Aliyi et al, 2023) (i.e., hyperproliferation, severe dysplasia) or cancer (Suzuki et al, 2021;Luo et al, 2019;Hirasawa et al, 2018;Iwagami et al, 2021) (i.e., adenocarcinoma, colorectal cancer, stomach cancer), but not suitable for fully-automated endoscopy examination. Only a few methods were intended for complete autonomous endoscopy examination; however, they detected only limited landmarks or abnormalities, additionally failed to achieve adequate accuracy and practical usability (Che et al, 2021;Tran et al, 2021;Ayyoubi Nezhad et al, 2022;Borgli et al, 2020). Che et al (2021) proposed a method for detecting anatomical landmarks in the lower GI tract from colonoscopy videos.…”

“…Only a few methods were intended for complete autonomous endoscopy examination; however, they detected only limited landmarks or abnormalities, additionally failed to achieve adequate accuracy and practical usability (Che et al, 2021;Tran et al, 2021;Ayyoubi Nezhad et al, 2022;Borgli et al, 2020). Che et al (2021) proposed a method for detecting anatomical landmarks in the lower GI tract from colonoscopy videos. They trained ResNet-101, a CNN-based network, to detect three landmarks, which achieved 92% accuracy.…”

“…They have detected ten landmarks with 97% accuracy, suitable automated esophagogastroduodenoscopy. These methods Che et al (2021);Tran et al (2021) were trained and tested using locally collected data. Ayyoubi Nezhad et al (2022) proposed a method for detecting landmarks from the entire GI tract.…”