ELNet:Automatic classification and segmentation for esophageal lesions using convolutional neural network

Abstract

Automatic and accurate esophageal lesion classification and segmentation is of great significance to clinically estimate the lesion status of esophageal disease and make suitable diagnostic schemes. Due to individual variations and visual similarities of lesions in shapes, colors and textures, current clinical methods remain subject to potential high-risk and time-consumption issues. In this paper, we propose an Esophageal Lesion Network (ELNet) for automatic esophageal lesion classification and segmentation using deep convolutional neural networks (DCNNs). The underlying method automatically integrates dual-view contextual lesion information to extract global features and local features for esophageal lesion classification of four esophageal image types (Normal, Inflammation, Barrett, and Cancer) and proposes lesion-specific segmentation network for automatic esophageal lesion annotation of three esophageal lesion types at pixel level. For established clinical large-scale database of 1051 white-light endoscopic images, ten-fold cross-validation is used in method validation. Experiment results show that the proposed framework achieves classification with sensitivity of 0.9034, specificity of 0.9718 and accuracy of 0.9628, and the segmentation with sensitivity of 0.8018, specificity of 0.9655 and accuracy of 0.9462. All of these indicate that our method enables an efficient, accurate and reliable esophageal lesion diagnosis in clinical.The main contributions of our work can be generalized as follows: 1 For the first time, proposed ELNet enables an automatically and reliably comprehensive esophageal lesions classification of four esophageal lesion types (Normal, Inflammation, Barrett, and Cancer) and lesion-specific segmentation from clinically white-light esophageal images to make suitable and repaid diagnostic schemes for clinicians. 2 A novel Dual-Stream network (DSN) is proposed for esophageal lesion classification. DSN automatically integrates dual-view contextual lesion information using two CNN streams to complementarily extract the global features from the holistic esophageal images and the local features from the lesion patches. 3 Lesion-specific esophageal lesion annotation with Segmentation Network with Classification (SNC) strategy is proposed to automatically annotate three lesion types (Inflammation, Barrett, Cancer) at pixel level to reduce the intra-class differences of esophageal lesions. 4 A clinically large-scale database esophageal database is established for esophageal lesions classification and segmentation. This database includes 1051 white-light esophageal images, which consists of endoscopic images in four different lesion types. Each image in this database has a classification label and its corresponding segmentation annotation.

Identification of Barrett's Esophagus in Endoscopic Images Using Deep Learning

“…However, the BE cases were relatively insu cient. In another research, neural networks were developed to segment 797 endoscopic images of cancer, BE, and in ammation cases [20]. Compared to the above works, our study dedicated to the automated identi cation and location of BE scopes in endoscopic images using DL.…”

“…Recently, DL has been gradually utilized in endoscopic image analysis of colon, stomach, and intestine, etc., with encouraging performance in identifying and diagnosing diseases such as tumors, polyps, and ulcers [16][17][18]. Meanwhile, several studies applied DL in the classi cation and segmentation of esophageal lesions [19][20][21][22][23][24][25]. However, there is still a lack of reports of developing DL methods dedicated to BE identi cation.…”

“…In one study, a migration-based learning approach was adopted to segment endoscopic images included cancer and BE [19]. In another recent study, a convolutional neural network (CNN) was used to segment endoscopic images of cancer, BE, and in ammation [20]. Because of the importance of early diagnosis of BE in the prevention of EAC, it is worth further investigating DL in BE diagnosis with a su cient BE sample size covering all BE types according to the Prague classi cation system [26].…”

Deep learning for diagnosis of precancerous lesions in upper gastrointestinal endoscopy: A review

“…After pre-processing and data augmentation, the average sensitivity, specificity and accuracy of the CNN model were 94.23%, 94.67% and 85.83%, respectively. Wu et al [ 39 ] developed a CNN-based framework named ELNet for automatic esophageal lesion ( i.e. EAC, BE and inflammation) classification and segmentation, the ELNet achieved a classification sensitivity of 90.34%, specificity of 97.18% and accuracy of 96.28%.…”

Multi-pathology detection and lesion localization in WCE videos by using the instance segmentation approach

“…Gastrointestinal tract has been subject of some studies using CNNs for identification purposes, such as cancer detection (Li et al, 2018) or colorectal polyp detection (Liu et al, 2016;Misawa et al, 2018;Urban et al, 2018;Zhang et al, 2019), and even using the powerful instance segmentation approach based on the YOLO-net (Zheng et al, 2018), however in this case only for classification purposes. It has also been used for segmentation and classification in esophageal images (Wu et al, 2021) or for classification of abnormal images in large datasets (Guo and Yuan, 2020). CNNs have also been used for gastrointestinal automatic diagnosis based on WCE data for polyp detection (Baopu Li et al, 2009), ulcer and bleeding classification (Liaqat et al, 2018), ulcer detection (Alaskar et al, 2019) and ulcer and erosion detection (Fan et al, 2018).…”