Recent Advancements in Multimodal Medical Image Fusion Techniques
for Better Diagnosis: An Overview

Haribabu, Maruturi; Guruviah, Velmathi; Yogarajah, Pratheepan

doi:10.2174/1573405618666220606161137

Cited by 6 publications

(2 citation statements)

References 111 publications

(168 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The integration of data from different modalities is on the rise in Artificial Intelligence (AI) fields, such as Machine Learning (ML) and Deep Learning (DL), known as multimodal data fusion. This research area, with applications integrating areas from natural language processing to computer vision and beyond, has driven applications in healthcare [4,5], integrating EHRs with medical images [6][7][8][9] or signals from wearable devices [10,11]. Multimodal fusion has also led to applications such as autonomous driving [12,13], environmental sciences applications combining different sensors and satellite data [14,15], as well as many other Internet of Things (IoT) applications and system improvements [16][17][18].…”

Section: Introductionmentioning

confidence: 99%

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

Restrepo,

Wu,

Vásquez-Venegas

et al. 2024

Preprint

View full text Add to dashboard Cite

In the big data era, integrating diverse data modalities poses significant challenges, particularly in complex fields like healthcare. This paper introduces a new process model for multimodal Data Fusion for Data Mining, integrating embeddings and the Cross-Industry Standard Process for Data Mining with the existing Data Fusion Information Group model. Our model aims to decrease computational costs, complexity, and bias while improving efficiency and reliability. We also propose "disentangled dense fusion," a novel embedding fusion method designed to optimize mutual information and facilitate dense inter-modality feature interaction, thereby minimizing redundant information.We demonstrate the model's efficacy through three use cases: predicting diabetic retinopathy using retinal images and patient metadata, domestic violence prediction employing satellite imagery, internet, and census data, and identifying clinical and demographic features from radiography images and clinical notes. The model achieved a Macro F1 score of 0.92 in diabetic retinopathy prediction, an R-squared of 0.854 and sMAPE of 24.868 in domestic violence prediction, and a macro AUC of 0.92 and 0.99 for disease prediction and sex classification, respectively, in radiological analysis. These results underscore the Data Fusion for Data Mining model's potential to significantly impact multimodal data processing, promoting its adoption in diverse, resource-constrained settings.

show abstract

Section: Introductionmentioning

confidence: 99%

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

Restrepo,

Wu,

Vásquez-Venegas

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…By adopting an ensemble approach, this method leverages the diversity and complementary nature of the individual networks, resulting in a more robust and accurate fusion outcome. Additionally, the ensemble enables us to address uncertainties associated with the fusion process by providing a measure of confidence for the final fused image [6].…”

Section: Introductionmentioning

confidence: 99%

An Ensemble Learning Approach for Multi-Modal Medical Image Fusion using Deep Convolutional Neural Networks

Godla,

Kavitha,

Ashok

et al. 2023

IJACSA

View full text Add to dashboard Cite

Medical image fusion plays a vital role in enhancing the quality and accuracy of diagnostic procedures by integrating complementary information from multiple imaging modalities. In this study, we propose an ensemble learning approach for multi-modal medical image fusion utilizing deep convolutional neural networks (DCNNs) to predict brain tumour. The proposed method aims to exploit the inherent characteristics of different modalities and leverage the power of CNNs for improved fusion results. The Generative Adversarial Network (GAN) strengthens the input images. The ensemble learning framework comprises two main stages. Firstly, a set of DCNN models is trained independently on the respective input modalities, extracting high-level features that capture modalityspecific information. Each DCNN model is fine-tuned to optimize its performance for fusion. Secondly, a fusion module is designed to aggregate the individual modality features and generate a fused image. The fusion module employs a weighted averaging technique to assign appropriate weights to the features based on their relevance and significance. The fused image obtained through this process exhibits enhanced spatial details and improved overall quality compared to the individual modalities. On a diversified dataset made up of multi-modal medical images, thorough tests are carried out to assess the efficacy of the suggested approach. The fusion images exhibit improved visual quality, enhanced feature representation, and better preservation of diagnostic information. The BRATS 2018 dataset, which contains Multi-Modal MRI images and patients' healthcare information were used. The proposed method also demonstrates robustness across different medical imaging modalities, highlighting its versatility and potential for widespread adoption in clinical practice.

show abstract

On the use of UDWT and fuzzy sets for medical image fusion

Tirupal,

Pandurangaiah,

Roy

et al. 2023

Multimed Tools Appl

View full text Add to dashboard Cite

Recent Advancements in Multimodal Medical Image Fusion Techniques for Better Diagnosis: An Overview

Cited by 6 publications

References 111 publications

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

An Ensemble Learning Approach for Multi-Modal Medical Image Fusion using Deep Convolutional Neural Networks

On the use of UDWT and fuzzy sets for medical image fusion

Contact Info

Product

Resources

About