Development and Validation of an Automatic Image-Recognition Endoscopic Report Generation System: A Multicenter Study

Abstract

INTRODUCTION: Conventional gastrointestinal (GI) endoscopy reports written by physicians are time consuming and might have obvious heterogeneity or omissions, impairing the efficiency and multicenter consultation potential. We aimed to develop and validate an image recognition–based structured report generation system (ISRGS) through a multicenter database and to assess its diagnostic performance. Methods: First, we developed and evaluated an ISRGS combining real-time video capture, site identification, lesion detection, subcharacteristics analysis, and structured report generation. White light and chromoendoscopy images from patients with GI lesions were eligible for study inclusion. A total of 46,987 images from 9 tertiary hospitals were used to train, validate, and multicenter test (6:2:2). Moreover, 5,699 images were prospectively enrolled from Qilu Hospital of Shandong University to further assess the system in a prospective test set. The primary outcome was the diagnosis performance of GI lesions in multicenter and prospective tests. Results: The overall accuracy in identifying early esophageal cancer, early gastric cancer, early colorectal cancer, esophageal varices, reflux esophagitis, Barrett’s esophagus, chronic atrophic gastritis, gastric ulcer, colorectal polyp, and ulcerative colitis was 0.8841 (95% confidence interval, 0.8775–0.8904) and 0.8965 (0.8883–0.9041) in multicenter and prospective tests, respectively. The accuracy of cecum and upper GI site identification were 0.9978 (0.9969–0.9984) and 0.8513 (0.8399–0.8620), respectively. The accuracy of staining discrimination was 0.9489 (0.9396–0.9568). The relative error of size measurement was 4.04% (range 0.75%–7.39%). DISCUSSION: ISRGS is a reliable computer-aided endoscopic report generation system that might assist endoscopists working at various hospital levels to generate standardized and accurate endoscopy reports ( http://links.lww.com/CTG/A485 ).

Diagnostic value of artificial intelligence-assisted endoscopy for chronic atrophic gastritis: a systematic review and meta-analysis

“…The study by Qu et al ( 39 ) did not use pathological findings as a diagnostic criterion but used expert consensus. There is a discrepancy between CAG diagnosis through endoscopic images and pathological results.…”

“…Five of the eight studies were retrospective ( 38 , 40 – 42 , 45 ), and three were prospective ( 29 , 43 , 44 ). All eight studies used deep-learning techniques: five used image-classification algorithms ( 38 , 41 , 42 , 44 , 45 ), one used an object-detection algorithm ( 39 ), one used a semantic-segmentation algorithm ( 43 ), and one used a combination of image classification and semantic segmentation ( 40 ). All studies were tested using static image models, and four studies used prospective videos to validate the models further ( 39 , 40 , 43 , 44 ).…”

“…All eight studies used deep-learning techniques: five used image-classification algorithms ( 38 , 41 , 42 , 44 , 45 ), one used an object-detection algorithm ( 39 ), one used a semantic-segmentation algorithm ( 43 ), and one used a combination of image classification and semantic segmentation ( 40 ). All studies were tested using static image models, and four studies used prospective videos to validate the models further ( 39 , 40 , 43 , 44 ). Regarding the type of endoscopy, four studies included only normal white-light endoscopy ( 38 , 40 – 42 ), three used enhanced endoscopy ( 39 , 44 , 45 ), and one did not specify the type of endoscopy ( 43 ).…”

“…All studies were tested using static image models, and four studies used prospective videos to validate the models further ( 39 , 40 , 43 , 44 ). Regarding the type of endoscopy, four studies included only normal white-light endoscopy ( 38 , 40 – 42 ), three used enhanced endoscopy ( 39 , 44 , 45 ), and one did not specify the type of endoscopy ( 43 ). Seven studies used pathology as the gold standard, and one used expert consensus as the decision criterion ( 39 ).…”

Application of deep learning in the real-time diagnosis of gastric lesion based on magnifying optical enhancement videos

“…Polyps can be identified in real time with 96% accuracy in screening colonoscopy (12). A multicenter study showed that C-WLI endoscopy can detect 80% of EGCs (13). AI shows an outstanding application in detection.…”