FusionGAN: A generative adversarial network for infrared and visible image fusion

Ma, Jiayi; Yu, Wei; Liang, Pengwei; Li, Chang; Jiang, Junjun

doi:10.1016/j.inffus.2018.09.004

Cited by 1,068 publications

(458 citation statements)

References 38 publications

Supporting

Mentioning

457

Contrasting

Unclassified

Order By: Relevance

“…Li et al [45] proposed a CNN network with the dense block structure to solve the infrared and visible image fusion problem. To improve the perceptual quality of the fused image, Ma [46] proposed a generative adversarial network (GAN), called FusionGAN, for infrared and visual image fusion.…”

Section: Multi-modal Image Fusionmentioning

confidence: 99%

Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion

Deng

Dragotti

2021

IEEE Trans. Pattern Anal. Mach. Intell.

111

View full text Add to dashboard Cite

In this paper, we propose a novel deep convolutional neural network to solve the general multi-modal image restoration (MIR) and multi-modal image fusion (MIF) problems. Different from other methods based on deep learning, our network architecture is designed by drawing inspirations from a new proposed multi-modal convolutional sparse coding (MCSC) model. The key feature of the proposed network is that it can automatically split the common information shared among different modalities, from the unique information that belongs to each single modality, and is therefore denoted with CU-Net, i.e., Common and Unique information splitting network. Specifically, the CU-Net is composed of three modules, i.e., the unique feature extraction module (UFEM), common feature preservation module (CFPM), and image reconstruction module (IRM). The architecture of each module is derived from the corresponding part in the MCSC model, which consists of several learned convolutional sparse coding (LCSC) blocks. Extensive numerical results verify the effectiveness of our method on a variety of MIR and MIF tasks, including RGB guided depth image super-resolution, flash guided non-flash image denoising, multi-focus and multi-exposure image fusion. Index Terms-Multi-modal image restoration, image fusion, multi-modal convolutional sparse coding ! • The authors are with the

show abstract

Section: Multi-modal Image Fusionmentioning

confidence: 99%

Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion

Deng

Dragotti

2021

IEEE Trans. Pattern Anal. Mach. Intell.

111

View full text Add to dashboard Cite

show abstract

“…In recent years, deep learning methods have been widely used to automatically learn features from raw data and make successful computer vision applications, 22,23 especially in HAR. [24][25][26] Nonetheless, deep learning methods usually need large-scale training set, which is difficult because of economic and technical limit.…”

Section: Introductionmentioning

confidence: 99%

Depth-based human activity recognition via multi-level fused features and fast broad learning system

Yao

Meng-ting

Chen

et al. 2020

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

Human activity recognition using depth videos remains a challenging problem while in some applications the available training samples is limited. In this article, we propose a new method for human activity recognition by crafting an integrated descriptor called multi-level fused features for depth sequences and devising a fast broad learning system based on matrix decomposition for classification. First, the surface normals are computed from original depth maps; the histogram of the surface normal orientations is obtained as a low-level feature by accumulating the contributions from normals, then a high-level feature is acquired by sparse coding and pooling on the aggregation of polynormals. After that, the principal component analysis is applied to the conjunction of the two-level features in order to obtain a lowdimensional and discriminative fused feature. At last, fast broad learning system based on matrix decomposition is proposed to accelerate the training process and enhance the classification results. The recognition results on three benchmark data sets show that our method outperforms the state-of-the-art methods in term of accuracy, especially when the number of training samples is small.

show abstract

“…Moreover, experimental results in complicated ambient conditions show that the proposed algorithm in this paper outperforms state-of-the-art algorithms in both qualitative and quantitative evaluations, and this study can extend to the field of other-type image fusion.Remote Sens. 2020, 12, 781 2 of 21 methods [16][17][18], deep learning-based methods [2,19,20] and other methods [21,22]. Next, the ideas of these methods are briefly introduced.The subspace-based method first projects high-dimensional source image into low-dimensional space, and then fuses the information contained in the subspace, such as principal component analysis (PCA) [10], independent component analysis (ICA) [11], robust principal component analysis (RPCA) [12] and so on.…”

mentioning

confidence: 99%

“…Deep learning-based methods are to imitate the behavioral perception mechanism of the human brain, which has strong adaptability and feature extraction ability [2]. However, this kind of method is computationally intensive and requires high hardware equipment [19,20]. In addition, there are other ideas and perspectives that inspire new image fusion method, such as entropy [21], total variation [22] and so on.With the development of computer vision technology, saliency-based methods have been successfully implemented to infrared and visible image fusion because it effectively utilizes the complementary information of the source image.…”

mentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Method for Infrared and Visual Images Fusion Based on Visual Attention Technique

et al. 2020

View full text Add to dashboard Cite

Infrared and visible image fusion technology provides many benefits for human vision and computer image processing tasks, including enriched useful information and enhanced surveillance capabilities. However, existing fusion algorithms have faced a great challenge to effectively integrate visual features from complex source images. In this paper, we design a novel infrared and visible image fusion algorithm based on visual attention technology, in which a special visual attention system and a feature fusion strategy based on the saliency maps are proposed. Special visual attention system first utilizes the co-occurrence matrix to calculate the image texture complication, which can select a particular modality to compute a saliency map. Moreover, we improved the iterative operator of the original visual attention model (VAM), a fair competition mechanism is designed to ensure that the visual feature in detail regions can be extracted accurately. For the feature fusion strategy, we use the obtained saliency map to combine the visual attention features, and appropriately enhance the tiny features to ensure that the weak targets can be observed. Different from the general fusion algorithm, the proposed algorithm not only preserve the interesting region but also contain rich tiny details, which can improve the visual ability of human and computer. Moreover, experimental results in complicated ambient conditions show that the proposed algorithm in this paper outperforms state-of-the-art algorithms in both qualitative and quantitative evaluations, and this study can extend to the field of other-type image fusion.

show abstract

FusionGAN: A generative adversarial network for infrared and visible image fusion

Cited by 1,068 publications

References 38 publications

Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion

Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion

Depth-based human activity recognition via multi-level fused features and fast broad learning system

An Efficient Method for Infrared and Visual Images Fusion Based on Visual Attention Technique

Contact Info

Product

Resources

About