Application of Deep Learning Models to Improve Ulcerative Colitis Endoscopic Disease Activity Scoring Under Multiple Scoring Systems

Abstract

Background & Aims Lack of clinical validation and inter-observer variability are two limitations of endoscopic assessment and scoring of disease severity in patients with Ulcerative Colitis. We developed a deep learning (DL) model to improve, accelerate and automate UC detection, and predict the Mayo Endoscopic Subscore (MES) and the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). Methods A total of 134 prospective videos (1,550,030 frames) were collected and those with poor quality were excluded. The frames were labeled by experts based on MES and UCEIS scores. The scored frames were used to create a preprocessing pipeline and train multiple convolutional neural networks (CNNs) with proprietary algorithms in order to filter, detect and assess all frames. These frames served as the input for the DL model, with the output being continuous scores for MES and UCEIS (and its components). A graphical user interface was developed to support both labeling video sections and displaying the predicted disease severity assessment by the AI from endoscopic recordings. Results Mean absolute error (MAE) and mean bias were used to evaluate the distance of continuous model's predictions from ground truth and its possible tendency to over/under-predict were excellent for MES and UCEIS. The quadratic weighted kappa used to compare the inter-rater agreement between experts’ labels and the model's predictions showed strong agreement (0.87, 0.88 frame-level, 0.88, 0.90 section-level and 0.90, 0.78 at video-level, for MES and UCEIS, respectively). Conclusions We present the first fully automated tool that improves the accuracy of the MES and UCEIS, reduces the time between video collection and review, and improves subsequent quality assurance and scoring.

A systematic review and meta-analysis of artificial intelligence-diagnosed endoscopic remission in ulcerative colitis

“…The remaining 53 articles were read in detail, of which 18 were included. 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 The literature screening process is illustrated in Figure 1 . Details of the included studies are presented in Table 1 .…”

Application of deep learning in the diagnosis and evaluation of ulcerative colitis disease severity

“…Second, while using video materials may better replicate real-world clinical scenarios and enhance CAD utility in clinical settings, current automated video analysis systems yield suboptimal accuracy, as evidenced in recent studies. 45 , 46 In general, there is potential for enhancing CAD systems using both still images and videos. Thus, further studies are needed, incorporating more real-world video content to corroborate and expand on our observations.…”

Present and future of endoscopy precision for inflammatory bowel disease

“…AI-enabled assessment of inflammation and prediction of outcome S EVERAL GROUPS HAVE developed deep-learning models to grade endoscopic disease activity objectively using MES, [73][74][75] UCEIS, 76,77 or both scores 78,79 and showed excellent diagnostic performance and strong concordance with experts.…”