Automated evaluation of colon capsule endoscopic severity of ulcerative colitis using ResNet50

Abstract

Capsule endoscopy has been widely used as a non-invasive diagnostic tool for small or large intestinal lesions. In recent years, automated lesion detection systems using machine learning have been devised. This study aimed to develop an automated system for capsule endoscopic severity in patients with ulcerative colitis along the entire length of the colon using ResNet50. Capsule endoscopy videos from patients with ulcerative colitis were collected prospectively. Each single examination video file was partitioned into four segments: the cecum and ascending colon, transverse colon, descending and sigmoid colon, and rectum. Fifty still pictures (576 × 576 pixels) were extracted from each partitioned video. A patch (128 × 128 pixels) was trimmed from the still picture at every 32-pixel-strides. A total of 739,021 patch images were manually classified into six categories: 0) Mayo endoscopic subscore (MES) 0, 1) MES1, 2) MES2, 3) MES3, 4) inadequate quality for evaluation, and 5) ileal mucosa. ResNet50, a deep learning framework, was trained using 483,644 datasets and validated using 255,377 independent datasets. In total, 31 capsule endoscopy videos from 22 patients were collected. The accuracy rates of the training and validation datasets were 0.992 and 0.973, respectively. An automated evaluation system for the capsule endoscopic severity of ulcerative colitis was developed. This could be a useful tool for assessing topographic disease activity, thus decreasing the burden of image interpretation on endoscopists.

A trial deep learning-based model for four-class histologic classification of colonic tumor from narrow band imaging

“…A patch-based CNN has been utilized for automated detection of a target area within a whole slide image in digital pathology 17,18 . This method has been recently applied for automated severity mapping along the entire colorectum in patients with ulcerative colitis from capsule endoscopy video les 19 , as it is considered to have an advantage when the object for classi cation is composed of topographically varying elements, such as severity or atypism. When reconstructing histologic maps in resected specimens, one often encounters topographic heterogeneity in the grade of dysplasia as well as invading depth.…”

A trial deep learning-based model for four-class histologic classification of colonic tumor from narrow band imaging

“…A patch-based CNN has been utilized for automated detection of a target area within a whole slide image in digital pathology 21 , 22 . This method has been recently applied for automated severity mapping along the entire colorectum in patients with ulcerative colitis from capsule endoscopy video files 23 , as it is considered to have an advantage when the object for classification is composed of topographically varying elements, such as severity or atypism. When reconstructing histologic maps in resected specimens, one often encounters topographic heterogeneity in the grade of dysplasia as well as invading depth.…”

Machine and deep learning in inflammatory bowel disease

“…As target treatment outcomes have evolved over time for IBD, several indices for scoring the severity of inflammation on endoscopy have been developed [16–20]; however, these metrics have not been widely adopted and require clinicians to manually calculate the scores [21]. Several studies have investigated the use of machine learning models to score bowel inflammation severity – particularly in ulcerative colitis – using traditional endoscopy images and video as well as capsule endoscopy footage [22,23,24 ▪ ,25]. In one such study, Iacucci et al developed a model, which predicted endoscopic and histologic remission in ulcerative colitis patients using white light endoscopy and virtual chromoendoscopy video data [23].…”