AT-GAN: A generative adversarial network with attention and transition for infrared and visible image fusion

Rao, Yujing; Wu, Dan; Han, Mina; Wang, Ting; Yang, Yang; Leí, Tao; Zhou, Chengjiang; Bai, Haicheng; Xing, Lin

doi:10.1016/j.inffus.2022.12.007

Cited by 48 publications

(17 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the recent developments in image fusion is the use of deep learning techniques. AT‐GAN 2 proposes a generative adversarial network (GAN) with intensity attention modules and semantic transition modules to explore key information in infrared and visible modals. Han et al 1 propose to achieve targeted level image with a scene texture attention module.…”

Section: Related Workmentioning

confidence: 99%

“…Detecting objects in fused images of visible and infrared modals is a significant task for many applications like traffic surveillance and military reconnaissance. However, in this area, most of research focus on proposing better image fusion methods, 1,2 or designing better infrared object detection. 3,4 In other words, the significance of object detection on fusion images is underrated.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Chang

Gao

et al. 2023

Engineering Reports

Self Cite

View full text Add to dashboard Cite

Object detection on fused images of visible and infrared modals is of great importance for many applications, for example, surveillance and rescue at low‐light conditions. However, current detectors have difficulty for robust fused image detection for mainly two reasons. First, objects are presented in various shapes and sizes, making some hard samples cannot be localized accurately. Second, the same object category in the fused images will have different appearance due to changing weather condition, temperature and intrinsic heat. Such a contradiction will degrade the classification task of a detection network, since it cannot merge commonalities and distinguish differences well. In this paper, we propose to reconstruct the detection pipeline of current detectors, and enhance the detection ability on difficult samples in fused images. Specifically, a Dilation Pyramid Network (DPN) is designed at the lateral connection to generate and aggregate features of various receptive field, without increasing pyramid layers. To strengthen the classification, a Semantic Category Attention Module (SCAM) is proposed to capture attention centers of semantics in fused images, rather than object centers. Abundant experiments on two fusion datasets show that the proposed method achieves a satisfying performance, and both modules can greatly improve current generic detectors on fused images.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Chang

Gao

et al. 2023

Engineering Reports

Self Cite

View full text Add to dashboard Cite

show abstract

“…10,29 Deep learning-based methods, such as RFNet, AT-GAN, and SemLA, utilize deep neural networks to promote multimodal image registration accuracy. 15,36,37 However, deep learning-based methods depend greatly on the quality of training data, which limits their performance in the registration of coral reefs with less texture.…”

Section: Introductionmentioning

confidence: 99%

“…For example, SIFT was utilized to guarantee preregistration results close to the ground truth, and mutual information was utilized during the fine-tuning process to achieve the most precise registration results 10 , 29 . Deep learning-based methods, such as RFNet, AT-GAN, and SemLA, utilize deep neural networks to promote multimodal image registration accuracy 15 , 36 , 37 . However, deep learning-based methods depend greatly on the quality of training data, which limits their performance in the registration of coral reefs with less texture.…”

Section: Introductionmentioning

confidence: 99%

Automatic registration method for medium-resolution remote sensing images of coral reefs with morphological information pairing and constrained iterative fining

Chen,

Pian,

Chen

et al. 2023

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

“…Saavedra et al (2022) combined the multithreshold method with region growing for image segmentation and detection. With the development of deep learning (Rao et al 2023;Xie et al 2023), it has also been applied to BP detection. Yang & Li (2019) proposed a masked region convolutional neural network model called GBP-MRCNN for the morphological classification of Gband bright points (GBPs).…”

Section: Introductionmentioning

confidence: 99%

Hybrid Detection Algorithm and Study on the Quantity and Brightness Evolution Characteristics of Photospheric Bright Point Groups

Bai,

Yang,

Zhao

et al. 2023

ApJ

View full text Add to dashboard Cite

Photospheric bright points (BPs), the smallest magnetic elements in the photosphere, are constantly moving and changing. Studying the characteristics of these small-scale strong magnetic fields with kilogauss magnitudes could be of significant importance for investigating the coronal heating problem. Compared to the study of a few specific BPs, investigating the collective features of BP groups can provide us with a better understanding of the overall characteristics of BPs. However, there is still a lack of research on the evolution of BP groups, and the detection algorithm of BPs still has a lot of space for improvement. We propose a hybrid BP detection model (HBD-Model) that combines traditional algorithms and deep learning to improve detection accuracy. Using the HBD-Model, we focus on studying the evolution characteristics of the quantity and brightness of BP groups at different brightness levels and how these characteristics differ between quiet and active regions. Results show that the activity of BP groups is not random or disorderly. In different brightness levels and regions, their quantity and brightness evolution exhibit periodic or complex changes.

show abstract

AT-GAN: A generative adversarial network with attention and transition for infrared and visible image fusion

Cited by 48 publications

References 34 publications

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Toward reliable fusion object detection based on dilated pyramid and semantic attention

Automatic registration method for medium-resolution remote sensing images of coral reefs with morphological information pairing and constrained iterative fining

Hybrid Detection Algorithm and Study on the Quantity and Brightness Evolution Characteristics of Photospheric Bright Point Groups

Contact Info

Product

Resources

About