Deep Transfer Learning and Radiomics Feature Prediction of Survival of Patients with High-Grade Gliomas

Abstract

BACKGROUND AND PURPOSE: Patient survival in high-grade glioma remains poor, despite the recent developments in cancer treatment. As new chemo-, targeted molecular, and immune therapies emerge and show promising results in clinical trials, image-based methods for early prediction of treatment response are needed. Deep learning models that incorporate radiomics features promise to extract information from brain MR imaging that correlates with response and prognosis. We report initial production of a combined deep learning and radiomics model to predict overall survival in a clinically heterogeneous cohort of patients with high-grade gliomas. MATERIALS AND METHODS: Fifty patients with high-grade gliomas from our hospital and 128 patients with high-grade glioma from The Cancer Genome Atlas were included. For each patient, we calculated 348 hand-crafted radiomics features and 8192 deep features generated by a pretrained convolutional neural network. We then applied feature selection and Elastic Net-Cox modeling to differentiate patients into long-and short-term survivors. RESULTS: In the 50 patients with high-grade gliomas from our institution, the combined feature analysis framework classified the patients into long-and short-term survivor groups with a log-rank test P value , .001. In the 128 patients from The Cancer Genome Atlas, the framework classified patients into long-and short-term survivors with a log-rank test P value of .014. For the mixed cohort of 50 patients from our institution and 58 patients from The Cancer Genome Atlas, it yielded a log-rank test P value of .035. CONCLUSIONS: A deep learning model combining deep and radiomics features can dichotomize patients with high-grade gliomas into long-and short-term survivors. ABBREVIATIONS: C-indices 4 concordance indices; CNN 4 convolutional neural network; GBM 4 glioblastoma multiforme; HGG 4 high-grade glioma; OS 4 overall survival; SE 4 spin-echo; TCGA 4 the Cancer Genome Atlas G lioblastoma multiforme (GBM), the largest diagnostic subcategory of high-grade glioma (HGG) and the most common malignant adult brain tumor, afflicts 12,000-13,000 new patients annually in the United States. GBM, comprising a genetically and phenotypically heterogeneous category of tumors, has a

Machine and Deep Learning Based Radiomics Models for Preoperative Prediction of Benign and Malignant Sacral Tumors

“…Previous studies have compared the performance of deep learning and radiomics in differentiating benign and malignant breast lesions (13,15), predicting lymph node metastases of breast cancer (14), identifying of spinal metastases originated from the lung and other cancers (16), predicting of survival of patients with high-grade gliomas (17), and predicting the (24) found that their DNN model was 80% accurate in predicting complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer, which was better than LR and SVM models. Due to the rarity of primary sacral tumors, only a few previous studies have identified sacral tumor types using machine learning methods (1,5,10).…”

“…Previous studies have compared the performance of deep learning and radiomics in differentiating benign and malignant breast lesions ( 13 , 15 ), predicting lymph node metastases of breast cancer ( 14 ), identifying of spinal metastases originated from the lung and other cancers ( 16 ), predicting of survival of patients with high-grade gliomas ( 17 ), and predicting the invasiveness risk of Stage-I lung adenocarcinomas ( 18 ). Dong et al.…”

“…Deep neural network (DNN), as a deep architecture, has shown excellent performance in classification tasks and is increasingly being used in various areas of cancer research (11,12). Early studies on the application of deep learning to the detection or classification of lesions have shown that it performs better than traditional techniques and even better than radiologists on some tasks (13)(14)(15)(16)(17)(18). Ren et al (19) proposed a novel manifold regularized classification DNN to enhance CT image-based lung nodule classification.…”

Machine Learning-Based Radiomics Predicting Tumor Grades and Expression of Multiple Pathologic Biomarkers in Gliomas

“…Also more recently, researchers have demonstrated achievements of deep learning (DL) in the image segmentation and glioma grades prediction (32)(33)(34)(35)(36)(37). Convolutional neural networks (CNNs) started outperforming other methods on several high-profile image analysis projects.…”

Cross-Organ, Cross-Modality Transfer Learning: Feasibility Study for Segmentation and Classification

“…Many researchers in the machine learning community have shown that ImageNet pretrained CNNs, like VGGNet [7] and ResNet, are effective to solve various target image recognition tasks, with fine-tuning of the network using their own datasets, or using it as fixed feature extractors. Similarly, previous studies in the medical imaging field have shown that transfer learning can solve their own medical imaging analysis problems by using off-the-shelf feature extracters (e.g., [8]- [13]), fine-tuning it with their own, or other publicly available, dataset (e.g., [14]- [20]), and using it as feature extracters after fine-tuning (e.g., [20]).…”