Multimodal Medical Image Fusion based on Deep Learning Neural Network for Clinical Treatment Analysis

Abstract

:Multimodal medical image fusion technique is one of the most significant and useful disease investigative techniques by deriving the complementary information from different multimodality medical images. This research paper, proposed an efficient multimodal medical image fusion approach based on deep learning convolutional neural networks (CNN) for fusion process. Computed Tomography (CT), Magnetic Resonance Imaging (MRI) and Positran Emission Tomography (PET) are the input multimodality medical images used for the experimental work. In the proposed technique, a siamese convolutional network is adopted to create a weight map which integrates the pixel movement information from two or more multimodality medical images. The medical image fusion process is carried out in a multiscale manner via medical image pyramids to be more reliable with human visual insight. In addition, a local comparison based strategy is applied to adaptively correct the fusion mode for the decomposed coefficients. An experimental result of proposed fusion techniques provides the best fused multimodal medical images of highest quality, shortest processing time and best visualization in both visual quality and objective assessment criteria.

Adaptive Multimodal Neuroimage Integration for Major Depression Disorder Detection

“…Even though previous studies have yielded promising performance, they often extract sMRI and fMRI features manually, which requires domainspecific knowledge (Shen et al, 2017). Several deep learning models of multimodal medical image fusion are proposed to employ multimodal neuroimaging data for brain disease diagnosis (Rajalingam and Priya, 2018). However, existing studies usually focus on combining feature representation of multiple modalities and ignore significant inter-modality heterogeneity (Huang et al, 2019).…”