Ensemble learning of multiview CNN models for survival time prediction of brain tumor patients using multimodal MRI scans

Abstract

Brain tumors have been one of the most common life-threatening diseases for all mankind. There have been huge efforts dedicated to the development of medical imaging techniques and radiomics to diagnose tumor patients quickly and efficiently. One of the main aims is to ensure that pre-operative overall survival time (OS) prediction is accurate. Recently, deep learning (DL) algorithms, and particularly convolutional neural networks (CNNs) achieved promising performances in almost all computer vision fields. CNNs demand large training datasets and high computational costs. However, curating large annotated medical datasets are difficult and resource-intensive. The performances of single learners are also unsatisfactory for small datasets. Thus, this study was conducted to improve the performance of CNN models on small volumetric datasets through developing a DL-based ensemble method for OS classification of brain tumor patients using multi-modal magnetic resonance images (MRI). First, we proposed Multi-View CNNs: OS classifiers based on representing the 3D MRI data as a set of 2D slices along all three planes (axial, sagittal, and coronal) and process them using 2D CNNs. Subsequently, the predicted probabilities by the Multi-View CNN models were fused using standard machine learning algorithms. The proposed approach was experimentally evaluated on 163 patients obtained from the BraTS'17 training dataset. Our best model achieved an AUC and accuracy values of 0.93 and 92.9%, respectively, on classifying patients with brain tumors into two OS groups, outperforming current state-of-the-art results. In addition, the FLAIR MRI modality yielded the best classification accuracy compared to other MRI modalities. Similarly, axial projections had the best classification performance compared to coronal and sagittal projections. Our findings may provide valuable insights for physicians in advancing treatment planning via noninvasive and accurate prediction of survival using only MRIs at the time of diagnosis.

DADFN: dynamic adaptive deep fusion network based on imaging genomics for prediction recurrence of lung cancer

“…The judgment condition is that when the newly added base classifier cannot improve the model accuracy, stop adding the base classifier in same layer. Mossa and Cevik (2021) proposed an integrated approach based on deep learning for overall survival (OS) classification of brain tumor patients using multimodal magnetic resonance images (MRI) to improve the performance of CNN model on small volume datasets.…”

Ensemble the recent architectures of deep convolutional networks for skin diseases diagnosis

“…In another CNN-based biomedical image processing application made by Mossa and Çevik in 2021, a predictive system for the survival times of patients with brain tumors was developed. 12 The 3D dataset of MRI images from 163 patients was insufficient for conventional image processing based on a single CNN. Instead, 2D segments were extracted from each axis of the 3D MRI scans and used to create image datasets for training several CNN models.…”

An Efficient End-to-End Deep Neural Network for Interstitial Lung Disease Recognition and Classification

“…However, these works achieved less accuracy and computationally expensive due to inappropriate feature selection. Recently, Deep Learning (DL) techniques have already shown superior performance in various computer vision issues, raising hopes in the area, including medical image processing [15], classification [16], segmentation [17,18], and detection [19,20]. Moreover, specific DL approaches called the pre-trained model of convolutional neural network (CNN), such as AlexNet [21], VGG-16 [22], ResNet-50 [23], were used to achieve competitive accuracy.…”