Infrared and visible image fusion via parallel scene and texture learning

Xu, Meilong; Tang, Linfeng; Zhang, Hao; Ma, Jiayi

doi:10.1016/j.patcog.2022.108929

Cited by 28 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This framework, even with a relatively small network size, proposes a network that still possesses a large receptive field. The literature [38] suggested a trainable hybrid network to enhance similar degradation problems, assessed the global content of low-light inputs using an encoderdecoder network, and output a new spatially variational RNN as edge flow, exhibiting favorable image fusion performance. Among these methods, some researchers put forward a fusion framework based on content and detail branches.…”

Section: Fusion Methods Based On Rnnsmentioning

confidence: 99%

OMOFuse: An Optimized Dual-Attention Mechanism Model for Infrared and Visible Image Fusion

Yuan,

2023

Mathematics

View full text Add to dashboard Cite

Infrared and visible image fusion aims to fuse the thermal information of infrared images and the texture information of visible images into images that are more in compliance with people’s visual perception characteristics. However, in the existing related work, the fused images have incomplete contextual information and poor fusion results. This paper presents a new image fusion algorithm—OMOFuse. At first, both the channel and spatial attention mechanisms are optimized by a DCA (dual-channel attention) mechanism and an ESA (enhanced spatial attention) mechanism. Then, an ODAM (optimized dual-attention mechanism) module is constructed to further improve the integration effect. Moreover, a MO module is used to improve the network’s feature extraction capability for contextual information. Finally, there is the loss function ℒ from the three parts of SSL (structural similarity loss), PL (perceptual loss), and GL (gap loss). Extensive experiments on three major datasets are performed to demonstrate that OMOFuse outperforms the existing image fusion methods in terms of quantitative determination, qualitative detection, and superior generalization capabilities. Further evidence of the effectiveness of our algorithm in this study are provided.

show abstract

Section: Fusion Methods Based On Rnnsmentioning

confidence: 99%

OMOFuse: An Optimized Dual-Attention Mechanism Model for Infrared and Visible Image Fusion

Yuan,

2023

Mathematics

View full text Add to dashboard Cite

show abstract

Section: Agricultural Protectionmentioning

confidence: 99%

“…Using texture learning and parallel scene, Xu and co‐workers present image fusion IR–vis approach (Figure 23c), leveraging detail branch and deep neural networks' content branch to concurrently gather various properties of the source images before reconstructing the combined image. [ 256 ] In images with visible insufficient light (00054N and 01042N), pedestrians cannot be detected and seen, and stationary autos (00689 N and 01061 N) cannot be recognized due to their little heat at night. Maintaining contrast and retaining structural texture detail, merged images provide improved detecting results, detected cars and pedestrians with improved confidence of their locations.…”

Section: Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Luminescence‐Based Infrared Thermal Sensors: Comprehensive Insights

Fahad,

Li,

Zhai

et al. 2023

Small

View full text Add to dashboard Cite

Recent chronological breakthroughs in materials innovation, their fabrication, and structural designs for disparate applications have paved transformational ways to subversively digitalize infrared (IR) thermal imaging sensors from traditional to smart. The noninvasive IR thermal imaging sensors are at the cutting edge of developments, exploiting the abilities of nanomaterials to acquire arbitrary, targeted, and tunable responses suitable for integration with host materials and devices, intimately disintegrate variegated signals from the target onto depiction without any discomfort, eliminating motional artifacts and collects precise physiological and physiochemical information in natural contexts. Highlighting several typical examples from recent literature, this review article summarizes an accessible, critical, and authoritative summary of an emerging class of advancement in the modalities of nano and micro‐scale materials and devices, their fabrication designs and applications in infrared thermal sensors. Introduction is begun covering the importance of IR sensors, followed by a survey on sensing capabilities of various nano and micro structural materials, their design architects, and then culminating an overview of their diverse application swaths. The review concludes with a stimulating frontier debate on the opportunities, difficulties, and future approaches in the vibrant sector of infrared thermal imaging sensors.

show abstract

“…DL-based methods can fuse infrared and visible images in an end-to-end way with their powerful nonlinear fitting ability [9]. DL methods include CNN-based methods, generative adversarial network (GAN)-based methods, transformer-based methods [21] and the other methods [22], [23]. CNN-based methods extract infrared and visible image features by designing parallel convolution kernels [24].…”

Section: Introductionmentioning

confidence: 99%

TFIV: Multigrained Token Fusion for Infrared and Visible Image via Transformer

Li,

Yang,

Bai

et al. 2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Infrared and visible image fusion aims to extract complementary features to synthesize a single fused image. Many methods employ convolutional neural networks (CNNs) to extract local features due to its translation invariance and locality. However, CNNs fail to consider the image's non-local self-similarity (NLss), though it can expand the receptive field by pooling operations, it still inevitably leads to information loss. In addition, the transformer structure extracts long-range dependence by considering the correlativity among all image patches, leading to information redundancy of such transformer-based methods. However, graph representation is more flexible than grid (CNN) or sequence (transformer structure) representation to address irregular objects, and graph can also construct the relationships among the spatially repeatable details or texture with far-space distance. Therefore, to address the above issues, it is significant to convert images into the graph space and thus adopt graph convolutional networks (GCNs) to extract NLss. This is because the graph can provide a fine structure to aggregate features and propagate information across the nearest vertices without introducing redundant information. Concretely, we implement a cascaded NLss extraction pattern to extract NLss of intra-and inter-modal by exploring interactions of different image pixels in intra-and inter-image positional distance. We commence by preforming GCNs on each intra-modal to aggregate features and propagate information to extract independent intra-modal NLss. Then, GCNs are performed on the concatenate intramodal NLss features of infrared and visible images, which can explore the cross-domain NLss of inter-modal to reconstruct the fused image. We progressively expand the kennel sizes and dilations to increase the receptive field of GCNs, when we extract intra-and inter-modal NLss. Ablation studies and extensive experiments illustrates the effectiveness and superiority of the proposed method on three datasets.

show abstract

Infrared and visible image fusion via parallel scene and texture learning

Cited by 28 publications

References 30 publications

OMOFuse: An Optimized Dual-Attention Mechanism Model for Infrared and Visible Image Fusion

OMOFuse: An Optimized Dual-Attention Mechanism Model for Infrared and Visible Image Fusion

Luminescence‐Based Infrared Thermal Sensors: Comprehensive Insights

TFIV: Multigrained Token Fusion for Infrared and Visible Image via Transformer

Contact Info

Product

Resources

About