GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion

Ma, Jiayi; Zhang, Hao; Shao, Zhenfeng; Liang, Pengwei; Xu, Han

doi:10.1109/tim.2020.3038013

Cited by 197 publications

(127 citation statements)

References 51 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…In order to verify the effectiveness of our FusionADA, we show some intuitive results from our work with 7 other state-of-the-art infrared and visible fusion methods, containing gradient transfer fusion (GTF) [ 24 ], fourth-order partial differential equations (FPDE) [ 25 ], hybrid multi-scale decomposition (HMSD) [ 26 ], DenseFuse [ 19 ], proportional maintenance of gradient and intensity (PMGI) [ 27 ], unified unsupervised image fusion (U2Fusion) [ 28 ], and generative adversarial network with multi-classification constraints (GANMcC) [ 29 ]. Among them, GTF, FPDE and HMSD are fusion methods based on the traditional framework, while DenseFuse, PMGI, U2Fusion and GANMcC are deep learning-based fusion methods.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images

Wang

Zhang

et al. 2021

Entropy

View full text Add to dashboard Cite

Obtaining key and rich visual information under sophisticated road conditions is one of the key requirements for advanced driving assistance. In this paper, a newfangled end-to-end model is proposed for advanced driving assistance based on the fusion of infrared and visible images, termed as FusionADA. In our model, we are committed to extracting and fusing the optimal texture details and salient thermal targets from the source images. To achieve this goal, our model constitutes an adversarial framework between the generator and the discriminator. Specifically, the generator aims to generate a fused image with basic intensity information together with the optimal texture details from source images, while the discriminator aims to force the fused image to restore the salient thermal targets from the source infrared image. In addition, our FusionADA is a fully end-to-end model, solving the issues of manually designing complicated activity level measurements and fusion rules existing in traditional methods. Qualitative and quantitative experiments on publicly available datasets RoadScene and TNO demonstrate the superiority of our FusionADA over the state-of-the-art approaches.

show abstract

Section: Experimental Results and Analysismentioning

confidence: 99%

Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images

Wang

Zhang

et al. 2021

Entropy

View full text Add to dashboard Cite

show abstract

“…Our method is based on the PyTorch framework and runs on a NVIDIA GTX1070 GPU. We evaluated the effectiveness and efficiency of our scheme by comparing with a number of recent CNN-based methods (i.e., PMGI [50], FGAN [37], DDcGAN [36], FusionDN [46], U2Fusion [45], RFN [19], GANMcC [38], Nest [16], DID [51] and Dense [15]).…”

Section: Training Detailsmentioning

confidence: 99%

Searching a Hierarchically Aggregated Fusion Architecture for Fast Multi-Modality Image Fusion

Liu

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Multi-modality image fusion refers to generating a complementary image that integrates typical characteristics from source images. In recent years, we have witnessed the remarkable progress of deep learning models for multi-modality fusion. Existing CNN-based approaches strain every nerve to design various architectures for realizing these tasks in an end-to-end manner. However, these handcrafted designs are unable to cope with the high demanding fusion tasks, resulting in blurred targets and lost textural details. To alleviate these issues, in this paper, we propose a novel approach, aiming at searching effective architectures according to various modality principles and fusion mechanisms. Specifically, we construct a hierarchically aggregated fusion architecture to extract and refine fused features from feature-level and object-level fusion perspectives, which is responsible for obtaining complementary target/detail representations. Then by investigating diverse effective practices, we composite a more flexible fusion-specific search space. Motivated by the collaborative principle, we employ a new search strategy with different principled losses and hardware constraints for sufficient discovery of components. As a result, we can obtain a task-specific architecture with fast inference time. Extensive quantitative and qualitative results demonstrate the superiority and versatility of our method against state-of-the-art methods. CCS CONCEPTS• Computing methodologies → Computer vision.

show abstract

“…Furthermore, Walds protocol [55] can not be applied to infrared-visible image fusion, so that there is no resolution degradation step for all infrared-visible image fusion. In many cases, both original visible and infrared images are used as supervising information such as [110]. How to provide supervising information sometimes depends on the application scenarios of the fusion.…”

Section: Infrared-visible Image Fusionmentioning

confidence: 99%

A Review on Remote Sensing Data Fusion with Generative Adversarial Networks (GAN)

Liu¹

2021

Preprint

View full text Add to dashboard Cite

In the past decades, remote sensing (RS) data fusion has always been an active research community. A large number of algorithms and models have been developed. Generative Adversarial Networks (GAN), as an important branch of deep learning, show promising performances in variety of RS image fusions. This review provides an introduction to GAN for remote sensing data fusion. We briefly review the frequently-used architecture and characteristics of GAN in data fusion and comprehensively discuss how to use GAN to realize fusion for homogeneous RS data, heterogeneous RS data, and RS and ground observation data. We also analyzed some typical applications with GAN-based RS image fusion. This review takes insight into how to make GAN adapt to different types of fusion tasks and summarizes the advantages and disadvantages of GAN-based RS data fusion. Finally, we discuss the promising future research directions and make a prediction on its trends.

show abstract

GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion

Cited by 197 publications

References 51 publications

Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images

Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images

Searching a Hierarchically Aggregated Fusion Architecture for Fast Multi-Modality Image Fusion

A Review on Remote Sensing Data Fusion with Generative Adversarial Networks (GAN)

Contact Info

Product

Resources

About