Multi-institutional Prognostic Modeling in Head and Neck Cancer: Evaluating Impact and Generalizability of Deep Learning and Radiomics

Abstract

Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging ({radiomics}). However, the development of prognostic models is complex as no modelling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records and pre-treatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and/or electronic medical record (EMR) data. The model with the highest accuracy used multitask learning on clinical data and tumour volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best-performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI/ML model utility and stronger validation frameworks.

RADCURE: An open‐source head and neck cancer CT dataset for clinical radiation therapy insights

“…The following datasets are included: AAPM-RT-MAC, 15,16 Cetuximab (RTOG 0522), 17,18 CPTAC-HNSCC, 19 HEAD-NECK-RADIOMICS-HN1, 4,6 QIN-HEADNECK, 20,21 Head-Neck-PET-CT, 5,7 TCGA-HNSC, 22 HNSCC, 23,24 and RADCURE. 13,14 F I G U R E 2 Heat maps showing the inclusion and completeness of data types across openly available datasets on TCIA 12 with comparable imaging to RADCURE. 13,14 Head-Neck-Radiomics-HN1, 4,6 HNSCC, 23,24 Head-Neck-PET-CT. Average slice thickness was 2.20 mm (range 2.00-3.00 mm) and the average number of slices was 183 (range 13−333).…”

“…13,14 F I G U R E 2 Heat maps showing the inclusion and completeness of data types across openly available datasets on TCIA 12 with comparable imaging to RADCURE. 13,14 Head-Neck-Radiomics-HN1, 4,6 HNSCC, 23,24 Head-Neck-PET-CT. Average slice thickness was 2.20 mm (range 2.00-3.00 mm) and the average number of slices was 183 (range 13−333). Imaging parameter information can be found in Table S1.…”

“…These repositories are instrumental in promoting reproducible science and the continuous evaluation of published research. In this manuscript, we introduce RADCURE [12][13][14] ; a singleinstitution dataset comprised of 3346 HNC computed tomography (CT) images with associated radiotherapy (RT) contours and outcomes data that is available on TCIA. RADCURE, with its significant patient count, not only reflects the diverse presentation of HNC patients treated at our institution but also contains the variability necessary to characterize this heterogenous disease site thoroughly.…”

Expert-level pediatric brain tumor segmentation in a limited data scenario with stepwise transfer learning

“…The progress in medical imaging techniques and computational methods have led to the development of various approaches for brain tumor segmentation, including machine learningbased methods, deep learning-based methods, and hybrid approaches (8)(9)(10). Recently, deep learning has emerged as a powerful tool in medical imaging, offering solutions to diverse clinical challenges (11)(12)(13)(14)(15). Auto-segmentation based on deep learning is thus a promissing approach for accurate and efficient brain tumor segmentation, including pLGGs (6,16,17), though distinct challenges remain.…”

Development and Validation of an Automated Image-Based Deep Learning Platform for Sarcopenia Assessment in Head and Neck Cancer

“…In past years, multiple deep learning models have been created and extensively used for medical imaging. [21][22][23][24][25][26] Although recent studies have applied deep learning techniques to determine skeletal muscle through abdominal CT scans on the L3 vertebral level, [27][28][29] few have been performed in head and neck cancer, a disease that has been increasing in prevalence and is known for its challenges in terms of patient vulnerability, treatment decisions, and long-term adverse effects. Recently, Naser et al 30 introduced a multistage deep learning approach for segmenting the C3 vertebral region using head and neck CT scans, which showed good model performance and a potential for predicting patient survival.…”