Deep Gradient Projection Networks for Pan-sharpening

Xu, Shuang; Zhang, Jiangshe; Zhao, Zixiang; Sun, Kai; Liu, Junmin

doi:10.1109/cvpr46437.2021.00142

Cited by 114 publications

(32 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evaluations on different satellites are carried out to evaluate the performance of PanFormer. We select 6 SOTA deep learning based methods including: PNN [3], MSDCNN [7], Pan-Net [5], PSGAN [15], DRPNN [6], and GPPNN [27], and 2 traditional methods including BDSD [11], and GS [26] for comparison.…”

Section: Comparisons With Sota Methodsmentioning

confidence: 99%

“…Params Time(s) PNN [3] 0.080M 0.0012 MSDCNN [7] 0.262M 0.0035 DRPNN [6] 1.639M 0.0031 PanNet [5] 0.078M 0.0032 PSGAN [15] 1.654M 0.0045 GPPNN [27] 0.120M 0.0211 PanFormer 1.530M 0.0468 the WorldView-3 testing set is given and only the parameters in the generator are count for those GAN based model. Table 3 shows that our method share a similar parameter count to DRPNN [6] and PSGAN [15].…”

Section: Modelmentioning

confidence: 99%

“…(b) PNN[3] (c) MSDCNN[7] (d) PanNet[5] (e) PSGAN[15] (f) DRPNN[6] (g) GPPNN[27] (h) PanFormerFig. The residuals between the HR MS image reconstructions and the ground truth from Fig.…”

mentioning

confidence: 99%

See 2 more Smart Citations

PanFormer: a Transformer Based Model for Pan-sharpening

Zhou¹,

Liu²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

Pan-sharpening aims at producing a high-resolution (HR) multi-spectral (MS) image from a low-resolution (LR) multispectral (MS) image and its corresponding panchromatic (PAN) image acquired by a same satellite. Inspired by a new fashion in recent deep learning community, we propose a novel Transformer based model for pan-sharpening. We explore the potential of Transformer in image feature extraction and fusion. Following the successful development of vision transformers, we design a two-stream network with the selfattention to extract the modality-specific features from the PAN and MS modalities and apply a cross-attention module to merge the spectral and spatial features. The pan-sharpened image is produced from the enhanced fused features. Extensive experiments on GaoFen-2 and WorldView-3 images demonstrate that our Transformer based model achieves impressive results and outperforms many existing CNN based methods, which shows the great potential of introducing Transformer to the pan-sharpening task. Codes are available at https://github.com/zhysora/PanFormer.

show abstract

Section: Comparisons With Sota Methodsmentioning

confidence: 99%

Section: Modelmentioning

confidence: 99%

See 1 more Smart Citation

PanFormer: a Transformer Based Model for Pan-sharpening

Zhou¹,

Liu²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Notably, they also achieve the second and third highest performance at the expense of massive model computation and storage. Additionally, the most comparable solution to ours, GPPNN (Xu et al 2021), is organized around the model-based unfolding principle and has comparable model parameters and flops reductions but inferior performance. This is due to powerful model learning's incapability without fully exploring the potential of different modalities.…”

Section: Complexity Analysismentioning

confidence: 99%

“…Different from SISR, GISR uses additional image to guide the super-resolution process. Typical GISR tasks include pansharpening (Masi et al 2016;Xu et al 2021;Cao et al 2021), depth image super-resolution (Kim et al 2021;Su et al 2019;Sun et al 2021), and magnetic resonance (MR) image super-resolution (Oktay et al 2016;Pham et al 2017; Recently, researchers have proposed a large number of guided image super-resolution (GISR) approaches. The main idea of these approaches is transferring structural details of the guidance image to the target image.…”

Section: Introductionmentioning

confidence: 99%

Memory-augmented Deep Unfolding Network for Guided Image Super-resolution

Zhou¹,

Yan²,

Pan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image. However, previous model-based methods mainly takes the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image, simply ignoring many non-local common characteristics between them. To alleviate this issue, we firstly propose a maximal a posterior (MAP) estimation model for GISR with two types of prior on the HR target image, i.e., local implicit prior and global implicit prior. The local implicit prior aims to model the complex relationship between the HR target image and the HR guidance image from a local perspective, and the global implicit prior considers the non-local auto-regression property between the two images from a global perspective. Secondly, we design a novel alternating optimization algorithm to solve this model for GISR. The algorithm is in a concise framework that facilitates to be replicated into commonly used deep network structures. Thirdly, to reduce the information loss across iterative stages, the persistent memory mechanism is introduced to augment the information representation by exploiting the Long short-term memory unit (LSTM) in the image and feature spaces. In this way, a deep network with certain interpretation and high representation ability is built. Extensive experimental results validate

show abstract