Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain

Computational and Mathematical Methods in Medicine

Yin

et al. 2020

119

The medical image fusion is the process of coalescing multiple images from multiple imaging modalities to obtain a fused image with a large amount of information for increasing the clinical applicability of medical images. In this paper, we attempt to give an overview of multimodal medical image fusion methods, putting emphasis on the most recent advances in the domain based on (1) the current fusion methods, including based on deep learning, (2) imaging modalities of medical image fusion, and (3) performance analysis of medical image fusion on mainly data set. Finally, the conclusion of this paper is that the current multimodal medical image fusion research results are more significant and the development trend is on the rise but with many challenges in the research field.

Section: Image Fusion Based On Deepmentioning

confidence: 99%

A Review of Multimodal Medical Image Fusion Techniques

Huang

Computational and Mathematical Methods in Medicine

Yin

et al. 2020

119

“…The core ideas of CNN are the local receptive field, weight sharing, and pooling layer, which greatly reduce the number of parameters in the neural network and effectively alleviates or avoids the overfitting phenomenon in the network model. Gradient descent method is adopted in CNN to minimize the loss function, and the weight parameters of the network is reversely adjusted, in which way the identification accuracy of the CNN model is improved through much iterative training [32].…”

Section: Convolutional Neural Network Modelmentioning

confidence: 99%

Improved Deep CNN with Parameter Initialization for Data Analysis of Near-Infrared Spectroscopy Sensors

Wang

Tian

et al. 2020

Sensors

Near-infrared (NIR) spectral sensors can deliver the spectral response of light absorbed by materials. Data analysis technology based on NIR sensors has been a useful tool for quality identification. In this paper, an improved deep convolutional neural network (CNN) with batch normalization and MSRA (Microsoft Research Asia) initialization is proposed to discriminate the tobacco cultivation regions using data collected from NIR sensors. The network structure is created with six convolutional layers and three full connection layers, and the learning rate is controlled by exponential attenuation method. One-dimensional kernel is applied as the convolution kernel to extract features. Meanwhile, the methods of L2 regularization and dropout are used to avoid the overfitting problem, which improve the generalization ability of the network. Experimental results show that the proposed deep network structure can effectively extract the complex characteristics inside the spectrum, which proves that it has excellent recognition performance on tobacco cultivation region discrimination, and it also demonstrates that the deep CNN is more suitable for information mining and analysis of big data.

“…Owing to the random initialized kernels, training the end-to-end model is unstable and difficult. An effective way to handle this issue is using a well-trained feature extraction model [33,34]. Thus, we choose pre-trained Resnet V1 [35] as the feature extraction layers.…”

Section: Generatormentioning

confidence: 99%

GANFuse: a novel multi-exposure image fusion method based on generative adversarial networks

Chen

et al. 2020

Neural Comput & Applic

In this paper, a novel multi-exposure image fusion method based on generative adversarial networks (termed as GANFuse) is presented. Conventional multi-exposure image fusion methods improve their fusion performance by designing sophisticated activity-level measurement and fusion rules. However, these methods have a limited success in complex fusion tasks. Inspired by the recent FusionGAN which firstly utilizes generative adversarial networks (GAN) to fuse infrared and visible images and achieves promising performance, we improve its architecture and customize it in the task of extreme exposure image fusion. To be specific, in order to keep content of extreme exposure image pairs in the fused image, we increase the number of discriminators differentiating between fused image and extreme exposure image pairs. While, a generator network is trained to generate fused images. Through the adversarial relationship between generator and discriminators, the fused image will contain more information from extreme exposure image pairs. Thus, this relationship can realize better performance of fusion. In addition, the method we proposed is an end-to-end and unsupervised learning model, which can avoid designing hand-crafted features and does not require a number of ground truth images for training. We conduct qualitative and quantitative experiments on a public dataset, and the experimental result shows that the proposed model demonstrates better fusion ability than existing multi-exposure image fusion methods in both visual effect and evaluation metrics.