A light-weight, efficient, and general cross-modal image fusion network

Fang, Aiqing; Zhao, Xinbo; Yang, Jiaqi; Qin, Beibei; Zhang, Yanning

doi:10.1016/j.neucom.2021.08.044

Cited by 20 publications

(3 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, its performance in the MEF task may be limited because it was trained only with a multi-focus image dataset. In [83], a general cross-modal image fusion network was presented, exploring the commonalities and characteristics of different fusion tasks. Different network structures were analyzed in terms of their impact on the quality and efficiency of image fusion.…”

Section: Supervised Methodsmentioning

confidence: 99%

Multi-Exposure Image Fusion Techniques: A Comprehensive Review

Liu

Song

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

Multi-exposure image fusion (MEF) is emerging as a research hotspot in the fields of image processing and computer vision, which can integrate images with multiple exposure levels into a full exposure image of high quality. It is an economical and effective way to improve the dynamic range of the imaging system and has broad application prospects. In recent years, with the further development of image representation theories such as multi-scale analysis and deep learning, significant progress has been achieved in this field. This paper comprehensively investigates the current research status of MEF methods. The relevant theories and key technologies for constructing MEF models are analyzed and categorized. The representative MEF methods in each category are introduced and summarized. Then, based on the multi-exposure image sequences in static and dynamic scenes, we present a comparative study for 18 representative MEF approaches using nine commonly used objective fusion metrics. Finally, the key issues of current MEF research are discussed, and a development trend for future research is put forward.

show abstract

Section: Supervised Methodsmentioning

confidence: 99%

Multi-Exposure Image Fusion Techniques: A Comprehensive Review

Liu

Song

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…A global cross-modal image fusion network was introduced in another paper [8] with the aim of evaluating the similarities and features exhibited in different fusion challenges. The impact of different network architectures on the effectiveness and quality of fusion was examined in the study.…”

Section: Literature Reviewmentioning

confidence: 99%

Deep Learning-Based Patch-Wise Illumination Estimation for Enhanced Multi-Exposure Fusion

AlZamili,

Danach,

Frikha

2023

IEEE Access

View full text Add to dashboard Cite

This article suggests a unique technique for multi-exposure fusion using convolutional neural networks (CNNs) for patch-wise illumination estimates. Multi-exposure fusion is a crucial component of enhancing image quality, particularly in circumstances with erratic lighting. Our proposed approach makes use of CNNs' capability to anticipate light levels inside specific image patches in order to accurately change exposure levels. We look at the theoretical foundations of our approach, emphasising the advantages of patch-wise estimation in capturing intricate lighting details. Additionally, we present experimental results demonstrating enhanced dynamic range expansion and image detail preservation, demonstrating that our methodology is more effective than conventional fusion methods. This study advances the state-of-the-art in multi-exposure fusion while also opening up new prospects for computational photography, surveillance, and computer vision applications.INDEX TERMS Convolutional neural network (CNN), dynamic range expansion, image enhancement, multi-exposure fusion, patch-wise illumination estimation.

show abstract

“…Notably, the fused NLRSIs are generally a fusion of optical images and night light images, and the intensity and morphological information of night light images are fused by comprehensively using the texture and spectrum characteristics of optical images to extract the potential information of human activities. However, owing to numerous complicated factors in the process of image fusion, fused images generally have many problems, such as spectral distortion and spatial detail distortion [ 20 , 21 ]. Despite the rapid development of remote sensing technology, existing research results cannot meet practical requirements, especially when different sensors are fused [ 22 , 23 ].…”

Section: Introductionmentioning

confidence: 99%

Color Night Light Remote Sensing Images Generation Using Dual-Transformation

Lu,

Zhou,

Huang

et al. 2024

Sensors

View full text Add to dashboard Cite

Traditional night light images are black and white with a low resolution, which has largely limited their applications in areas such as high-accuracy urban electricity consumption estimation. For this reason, this study proposes a fusion algorithm based on a dual-transformation (wavelet transform and IHS (Intensity Hue Saturation) color space transform), is proposed to generate color night light remote sensing images (color-NLRSIs). In the dual-transformation, the red and green bands of Landsat multi-spectral images and “NPP-VIIRS-like” night light remote sensing images are merged. The three bands of the multi-band image are converted into independent components by the IHS modulated wavelet transformed algorithm, which represents the main effective information of the original image. With the color space transformation of the original image to the IHS color space, the components I, H, and S of Landsat multi-spectral images are obtained, and the histogram is optimally matched, and then it is combined with a two-dimensional discrete wavelet transform. Finally, it is inverted into RGB (red, green, and blue) color images. The experimental results demonstrate the following: (1) Compared with the traditional single-fusion algorithm, the dual-transformation has the best comprehensive performance effect on the spatial resolution, detail contrast, and color information before and after fusion, so the fusion image quality is the best; (2) The fused color-NLRSIs can visualize the information of the features covered by lights at night, and the resolution of the image has been improved from 500 m to 40 m, which can more accurately analyze the light of small-scale area and the ground features covered; (3) The fused color-NLRSIs are improved in terms of their MEAN (mean value), STD (standard deviation), EN (entropy), and AG (average gradient) so that the images have better advantages in terms of detail texture, spectral characteristics, and clarity of the images. In summary, the dual-transformation algorithm has the best overall performance and the highest quality of fused color-NLRSIs.

show abstract

A light-weight, efficient, and general cross-modal image fusion network

Cited by 20 publications

References 51 publications

Multi-Exposure Image Fusion Techniques: A Comprehensive Review

Multi-Exposure Image Fusion Techniques: A Comprehensive Review

Deep Learning-Based Patch-Wise Illumination Estimation for Enhanced Multi-Exposure Fusion

Color Night Light Remote Sensing Images Generation Using Dual-Transformation

Contact Info

Product

Resources

About