Videomics of the Upper Aero-Digestive Tract Cancer: Deep Learning Applied to White Light and Narrow Band Imaging for Automatic Segmentation of Endoscopic Images

Abstract

IntroductionNarrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images.Materials and MethodsA dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets.Results219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%.ConclusionSegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.

Artificial Intelligence for Upper Aerodigestive Tract Endoscopy and Laryngoscopy: A Guide for Physicians and State‐of‐the‐Art Review

“…Particularly, Azam et al developed an automatic segmentation model suitable for NBI and WL laryngeal cancer endoscopic images (DSC = 0.814). Moreover, they tested the same model on 2 external datasets of the oral cavity and oropharynx carcinomas obtaining comparable results (DSC = 0.75‐0.87) 38 . This is particularly interesting as it shows that only 1 DL model, if adequately trained, can comprehensively segment lesions in the whole UADT.…”

“…Moreover, they tested the same model on 2 external datasets of the oral cavity and oropharynx carcinomas obtaining comparable results (DSC = 0.75-0.87). 38 This is particularly interesting as it shows that only 1 DL model, if adequately trained, can comprehensively segment lesions in the whole UADT.…”

The tumor ecosystem in head and neck squamous cell carcinoma and advances in ecotherapy

“…Recent advances in data management technologies, such as artificial intelligence and deep learning, are important for visualizing the dynamics of the tumor ecosystem and predicting cancer treatment [ 170 – 173 ]. Specifically, the deep learning segmentation model designed by convolutional neural networks can automatically segment the tumor, so that more accurate biopsies can be obtained for the research of its ecosystem in OSCC [ 174 ].…”

Accuracy of narrow band imaging for detecting the malignant transformation of oral potentially malignant disorders: A systematic review and meta-analysis

“…At present, various non-invasive early diagnostic methods are emerging and gaining significant attention in dental practice (7,8). Of note, narrow band imaging (NBI; Olympus Medical System Corporation, Tokyo, Japan), as a novel optical method, has recently been applied in the detection of dysplastic or malignant lesions, including those of the oropharynx and nasopharynx, and has exhibited sound performance in diagnosing malignant lesions of the digestive system, such as esophageal cancer (9)(10)(11). As for its mechanisms, under NBI mode, blue and green lights whose wavelengths are at 415 and 540 nm, are able to penetrate the mucosal surface and are then absorbed by the surficial blood vessels; thus, the vessels present as dark blue or brown (Figure 1).…”