Multi-modal fusion deep transfer learning for accurate brain tumor classification using magnetic resonance imaging images

Abstract

Early identification and treatment of brain tumors depend critically on accurate classification. Accurate brain tumor classification in medical imaging is essential for clinical decisions and individualized treatment plans. This paper introduces a novel method for classifying brain tumors called multimodal fusion deep transfer learning (MMFDTL) using original, contoured, and annotated magnetic resonance imaging (MRI) images to showcase its capabilities. The MMFDTL can capture complex tumor features frequently missed in analyzing individual modalities. The MMFDTL model employs three deep learning models for extracting features VGG16, Inception V3, and ResNet 50. The accuracy rate improves when combined with decision based multimodal fusion. It produces impressive outcomes of sensitivity 92.96%, specificity 98.54%, precision 93.60%, accuracy 98.80%, F1-score 93.26%, and kappa 91.86%. This research can improve medical imaging and brain tumor analysis through its multi modal fusion approach. It could give healthcare practitioners vital insights for personalized treatment plans and informed decision making.