Two Stages Pan-Sharpening Details Injection Approach Based on Very Deep Residual Networks

Benzenati, Tayeb; Kallel, A.; Kessentini, Yousri

doi:10.1109/tgrs.2020.3019835

Cited by 18 publications

(11 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, the concept of featuredomain injection gains is introduced for the first time. For detail injection, traditional MRA methods (e.g., Generalized Laplacian Pyramid [17]) have given a well-defined formula,…”

Section: Adaptive Detail Injection Modulementioning

confidence: 99%

“…Unfortunately, most of the existing first-rate deep learning methods ignore this thought and tend to be insufficient in controlling biased details, resulting in the spectral distortion problem. Some works [17] attempt to use CNN to regress the values of injection gains to deal with the spectral distortion problem. However, these works unexpectedly lead to the spatial distortion problem owing to rough injection gains estimation in the image domain, rather than in the feature domain.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Detail Injection-Based Feature Pyramid Network for Pan-Sharpening

Sun¹,

Zhang²,

Yuan³

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Many remarkable works have been proposed to deal with distortions problems in image fusion to date. However, the spectral distortion and the spatial distortion cannot always be well addressed at the same time. To deal with this, we propose an Adaptive Feature Pyramid Network (AFPN) to efficiently embed an Adaptive Detail Injection (ADI) module at different scales. Feature-domain injection gains are proposed in the ADI module to adaptively modulate spatial information and guide a refined detail injection. Furthermore, we propose a texture loss function to further guide our model to learn detail perception in each band. Experiments on QuickBird and GaoFen-1 datasets show that our method achieves superior performance and produces visually pleasing fusion images. Our code is available at https://github.com/yisun98/AFPN .

show abstract

Section: Adaptive Detail Injection Modulementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Adaptive Detail Injection-Based Feature Pyramid Network for Pan-Sharpening

Sun¹,

Zhang²,

Yuan³

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…Wei, Yuan, Shen, and Zhang (2017) used residual learning to develop a very deep CNN with 11 convolutional layers to improve the accuracy of pan-sharpening fusion. To reconstruct the spatial details in upsampled multispectral images, Benzenati, Kallel, and Kessentini (2020) and Liu et al (2020a) proposed two-stage approaches. In the first stage, CNNs captured mid-level and high-level spatial features from PAN images.…”

Section: Data Fusionmentioning

confidence: 99%

Deep learning for processing and analysis of remote sensing big data: a technical review

2021

View full text Add to dashboard Cite

In recent years, the rapid development of Earth observation technology has produced an increasing growth in remote sensing big data, posing serious challenges for effective and efficient processing and analysis. Meanwhile, there has been a massive rise in deeplearning-based algorithms for remote sensing tasks, providing a large opportunity for remote sensing big data. In this article, we initially summarize the features of remote sensing big data. Subsequently, following the pipeline of remote sensing tasks, a detailed and technical review is conducted to discuss how deep learning has been applied to the processing and analysis of remote sensing data, including geometric and radiometric processing, cloud masking, data fusion, object detection and extraction, landuse/cover classification, change detection and multitemporal analysis. Finally, we discussed technical challenges and concluded directions for future research in deep-learning-based applications for remote sensing big data.

show abstract

“…However, all of them are generally based on handcrafted features, with limited capacity to reconstruct the missing information in the MS images. Very recently, to overcome the aforementioned shortcomings, researchers focus on exploiting the powerful feature representation capability of convolution neural networks (CNNs) to construct numerous CNNs-based pan-sharpening methods (Wang et al 2021a,b;Xu et al 2021a;Peng et al 2021;Benzenati, Kallel, and Kessentini 2021;Hu et al 2021;Liu et al 2020;Xu et al 2021b;Cai and Huang 2021), which outperforms previous state-of-the-art methods by a large margin. However, existing CNN-based methods remain some limitations: 1) lacking the modeling of long-range dependency owing to the local neighbor reception characterize of convolution operator, 2) ineffective feature extraction and fusion.…”

Section: Introductionmentioning

confidence: 99%

Pan-Sharpening with Customized Transformer and Invertible Neural Network

Zhou

Huang

Fang

et al. 2022

AAAI

View full text Add to dashboard Cite

In remote sensing imaging systems, pan-sharpening is an important technique to obtain high-resolution multispectral images from a high-resolution panchromatic image and its corresponding low-resolution multispectral image. Owing to the powerful learning capability of convolution neural network (CNN), CNN-based methods have dominated this field. However, due to the limitation of the convolution operator, long-range spatial features are often not accurately obtained, thus limiting the overall performance. To this end, we propose a novel and effective method by exploiting a customized transformer architecture and information-lossless invertible neural module for long-range dependencies modeling and effective feature fusion in this paper. Specifically, the customized transformer formulates the PAN and MS features as queries and keys to encourage joint feature learning across two modalities while the designed invertible neural module enables effective feature fusion to generate the expected pan-sharpened results. To the best of our knowledge, this is the first attempt to introduce transformer and invertible neural network into pan-sharpening field. Extensive experiments over different kinds of satellite datasets demonstrate that our method outperforms state-of-the-art algorithms both visually and quantitatively with fewer parameters and flops. Further, the ablation experiments also prove the effectiveness of the proposed customized long-range transformer and effective invertible neural feature fusion module for pan-sharpening.

show abstract

Two Stages Pan-Sharpening Details Injection Approach Based on Very Deep Residual Networks

Cited by 18 publications

References 36 publications

Adaptive Detail Injection-Based Feature Pyramid Network for Pan-Sharpening

Adaptive Detail Injection-Based Feature Pyramid Network for Pan-Sharpening

Deep learning for processing and analysis of remote sensing big data: a technical review

Pan-Sharpening with Customized Transformer and Invertible Neural Network

Contact Info

Product

Resources

About