Multi-Institutional Validation of Deep Learning for Pretreatment Identification of Extranodal Extension in Head and Neck Squamous Cell Carcinoma

Abstract

PURPOSE Extranodal extension (ENE) is a well-established poor prognosticator and an indication for adjuvant treatment escalation in patients with head and neck squamous cell carcinoma (HNSCC). Identification of ENE on pretreatment imaging represents a diagnostic challenge that limits its clinical utility. We previously developed a deep learning algorithm that identifies ENE on pretreatment computed tomography (CT) imaging in patients with HNSCC. We sought to validate our algorithm performance for patients from a diverse set of institutions and compare its diagnostic ability to that of expert diagnosticians. METHODS We obtained preoperative, contrast-enhanced CT scans and corresponding pathology results from two external data sets of patients with HNSCC: an external institution and The Cancer Genome Atlas (TCGA) HNSCC imaging data. Lymph nodes were segmented and annotated as ENE-positive or ENE-negative on the basis of pathologic confirmation. Deep learning algorithm performance was evaluated and compared directly to two board-certified neuroradiologists. RESULTS A total of 200 lymph nodes were examined in the external validation data sets. For lymph nodes from the external institution, the algorithm achieved an area under the receiver operating characteristic curve (AUC) of 0.84 (83.1% accuracy), outperforming radiologists’ AUCs of 0.70 and 0.71 ( P = .02 and P = .01). Similarly, for lymph nodes from the TCGA, the algorithm achieved an AUC of 0.90 (88.6% accuracy), outperforming radiologist AUCs of 0.60 and 0.82 ( P < .0001 and P = .16). Radiologist diagnostic accuracy improved when receiving deep learning assistance. CONCLUSION Deep learning successfully identified ENE on pretreatment imaging across multiple institutions, exceeding the diagnostic ability of radiologists with specialized head and neck experience. Our findings suggest that deep learning has utility in the identification of ENE in patients with HNSCC and has the potential to be integrated into clinical decision making.

Identification of Clinical and Socioeconomic Predictors of Adjuvant Therapy after Trans-Oral Robotic Surgery in Patients with Oropharyngeal Squamous Cell Carcinoma

“…Careful review of preoperative imaging and incorporation of PET/CT imaging may improve our ability to detect occult nodal disease and should continue to be explored as a strategy to more accurately stage these patients and their optimal selection for TORS treatment. In addition, artificial and deep learning algorithms have been shown to have excellent accuracy at predicting extra-nodal extension in two external data sets in Head and Neck Squamous Cell Carcinoma (HNSCC) patients and represent a promising tool to improve patient selection for TORS [17]. In this study, the deep learning algorithm was better able to reliably predict ECE compared to radiology review.…”

Applications of radiomics in precision diagnosis, prognostication and treatment planning of head and neck squamous cell carcinomas

“…Random forest ML classifiers were trained and yielded an AUC (95% confidence interval) of 0.88 (0.81-0.95) for the detection of ENE and 0.91 (0.86-0.97) for nodal metastasis detection in an independent test set of 131 lymph nodes; whereasbeing the methodological focus of the studya deep neural network yielded an AUC performance of 0.91 (0.85-0.97) and 0.91 (0.86-0.96) for ENE and [106]. The deep neural network model generalized well to an external test set, outperforming radiologists in ENE classification [107]. Of note [106,107], there was no significant difference in performance of deep neural networks (exploratory radiomics) over (preset conventional) radiomic analysis in detection of ENE.…”

“…The deep neural network model generalized well to an external test set, outperforming radiologists in ENE classification [107]. Of note [106,107], there was no significant difference in performance of deep neural networks (exploratory radiomics) over (preset conventional) radiomic analysis in detection of ENE. They highlight the potential quantitative imaging may possess for augmenting radiologist performance and guiding HNSCC treatment.…”

“…In clinical practice, ENE is ascertained by pathology review after neck dissection, whereas radiographical identification remains challenging [104,105]. Kann et al developed [106] and validated [107] quantitative imaging tools for pre-operative detection of ENE: the group segmented 653 nodes in total (380 negative, 153 without ENE and 120 nodes with ENE) on contrast-enhanced CT scans and extracted 99 radiomic features [106]. Random forest ML classifiers were trained and yielded an AUC (95% confidence interval) of 0.88 (0.81-0.95) for the detection of ENE and 0.91 (0.86-0.97) for nodal metastasis detection in an independent test set of 131 lymph nodes; whereasbeing the methodological focus of the studya deep neural network yielded an AUC performance of 0.91 (0.85-0.97) and 0.91 (0.86-0.96) for ENE and [106].…”

Emerging Concepts and Novel Strategies in Radiation Therapy for Laryngeal Cancer Management

“…Machine learning may also be useful in identifying and developing normal tissue complication probability (NTCP) models [ 72 ]. Kann et al [ 73 ] used deep-learning algorithms to identify pretreatment extranodal extensions (ENEs) in the SCCs of the head and neck, and compared their own performance against two board-certified neuroradiologists. Preoperative, contrast-enhanced CT scans and pathology results from two external data sets were employed for analysis.…”