Integrating deep and radiomics features in cancer bioimaging

Multi-Level Multi-Modality Fusion Radiomics: Application to PET and CT Imaging for Prognostication of Head and Neck Cancer

“…Their study only made use of CT images and not PET images, and thus our study sheds light on the relative value of and optimal approaches to fusing information from both modalities. As for the study by Bizzego et al [35], they concluded that combining radiomics and deep learning features from both PET and CT images outperformed using only one feature type or single modality. The authors only considered prediction of recurrence, and reported Matthews Correlation Coefficient (MCC) of 0.748 instead of AUC or C-index, which is not directly comparable with our C-index of 0.67.…”

“…Since four centers were involved in the dataset, apart from one partition adopted in previous studies [33]- [35] (center 2 and center 3 used for training, and center 1 and center 4 used for testing, noted as CER 23 vs. 14), we additionally investigated six other kinds of partitions (CER 12 vs. 34, CER 13 vs. 24, CER 123 vs. 4, CER 124 vs. 3, CER 134 vs. 2 and CER 234 vs 1) by ensuring training patients to be more than testing patients, while those partitions having less training patients than testing patients were excluded. Three endpoints recurrence-free survival (RFS), metastasis-free survival (MFS) and overall survival (OS) were considered.…”

“…Three previous studies [33]- [35] used center HGJ and CHUS for training, and center HMR and CHUM for testing (i.e., CER 23 vs. 14 in our study); thus, below comparison is based on this partition. In comparison with the model reported by Vallières et al [33], our model showed lower C-index for prediction of MFS (0.82 vs. 0.88) and OS (0.70 vs. 0.76) and comparable C-index (0.67 vs. 0.67) for prediction of RFS.…”

Integrative Network Fusion: A Multi-Omics Approach in Molecular Profiling

“…In its current version, the INF framework supports the integration of two or more one-dimensional omics layers. As part of our future effort we will add support for the integration of medical imaging layers, for example leveraging the extraction of histopathological features from whole slide images by deep learning ( 10 ) or using radiomics or deep features from radiological images ( 11 ). In both cases, further issues will emerge from the interactions between the omics and the non-omics data, needing particular care in the integration ( 12 ).…”

“…The INF framework is currently designed to integrate an arbitrary number of one-dimensional omics layers. We plan to further extend the framework by enabling the integration of histopathological features extracted from whole slide images ( 10 ) or deep features from radiological images ( 11 ) extracted by deep neural network architectures, carefully addressing all potential caveats ( 12 ).…”