LAGConv: Local-Context Adaptive Convolution Kernels with Global Harmonic Bias for Pansharpening

Jin, Zi-Rong; Zhang, Tian-Jing; Jiang, Tai-Xiang; Vivone, Gemine; Deng, Liang-Jian

doi:10.1609/aaai.v36i1.19996

Cited by 36 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most supervised methods seek to enhance the pan-sharpening effect by exploring more advanced network architectures. Some high-performance networks include PanNet (Yang et al 2017), MSDCNN (Yuan et al 2018), DiCNN (He et al 2019), SRPPPNN (Cai and Huang 2020), GPPNN (Xu et al 2021), LAGConv (Jin et al 2022), SFIIN (Zhou et al 2022c), among others (Zhou et al 2022b). Due to the lack of ground-truth for pan-sharpening, supervised methods are typically trained using reduced-resolution data.…”

Section: Supervised Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration

Lin,

Dong,

Ding

et al. 2024

AAAI

View full text Add to dashboard Cite

Pan-sharpening is a task that aims to super-resolve the low-resolution multispectral (LRMS) image with the guidance of a corresponding high-resolution panchromatic (PAN) image. The key challenge in pan-sharpening is to accurately modeling the relationship between the MS and PAN images. While supervised deep learning methods are commonly employed to address this task, the unavailability of ground-truth severely limits their effectiveness. In this paper, we propose a mutually guided detail restoration method for unsupervised pan-sharpening. Specifically, we treat pan-sharpening as a blind image deblurring task, in which the blur kernel can be estimated by a CNN. Constrained by the blur kernel, the pan-sharpened image retains spectral information consistent with the LRMS image. Once the pan-sharpened image is obtained, the PAN image is blurred using a pre-defined blur operator. The pan-sharpened image, in turn, is used to guide the detail restoration of the blurred PAN image. By leveraging the mutual guidance between MS and PAN images, the pan-sharpening network can implicitly learn the spatial relationship between the two modalities. Extensive experiments show that the proposed method significantly outperforms existing unsupervised pan-sharpening methods.

show abstract

Section: Supervised Methodsmentioning

confidence: 99%

“…Recently, deep learning methods (Jin et al 2022) have also been applied to pan-sharpening due to their powerful feature extraction capabilities. By utilizing a large amount of labeled data for training, supervised pan-sharpening methods significantly outperform traditional methods in restoring spectral and spatial details.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration

Lin,

Dong,

Ding

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…The traditional methods included GSA (Aiazzi, Baronti, and Selva 2007), FUSE (Wei, Dobigeon, and Tourneret 2015) and CNMF (Yokoya, Yairi, and Iwasaki 2011). The Deep Learningbased methods include PSRT (Deng et al 2023), LightNet (Chen et al 2022), LAGC-NET (Jin et al 2022), MoG-DCN (Dong et al 2021) and SSR-NET (Zhang et al 2021). Four widely used indexes are used for quantitative evaluation, including peak signal-to noise ratio (PSNR), spectral angle mapper (SAM), root mean squared error (RMSE), and erreur relative global adimensionnelle de synthese (ERGAS).…”

Section: Competing Methods and Evaluation Metricsmentioning

confidence: 99%

S2CycleDiff: Spatial-Spectral-Bilateral Cycle-Diffusion Framework for Hyperspectral Image Super-resolution

Qu,

He,

Dong

et al. 2024

AAAI

View full text Add to dashboard Cite

Hyperspectral image super-resolution (HISR) is a technique that can break through the limitation of imaging mechanism to obtain the hyperspectral image (HSI) with high spatial resolution. Although some progress has been achieved by existing methods, most of them directly learn the spatial-spectral joint mapping between the observed images and the target high-resolution HSI (HrHSI), failing to fully reserve the spectral distribution of low-resolution HSI (LrHSI) and the spatial distribution of high-resolution multispectral imagery (HrMSI). To this end, we propose a spatial-spectral-bilateral cycle-diffusion framework (S2CycleDiff) for HISR, which can step-wise generate the HrHSI with high spatial-spectral fidelity by learning the conditional distribution of spatial and spectral super-resolution processes bilaterally. Specifically, a customized conditional cycle-diffusion framework is designed as the backbone to achieve the spatial-spectral-bilateral super-resolution by repeated refinement, wherein the spatial/spectral guided pyramid denoising (SGPD) module seperately takes HrMSI and LrHSI as the guiding factors to achieve the spatial details injection and spectral correction. The outputs of the conditional cycle-diffusion framework are fed into a complementary fusion block to integrate the spatial and spectral details to generate the desired HrHSI. Experiments have been conducted on three widely used datasets to demonstrate the superiority of the proposed method over state-of-the-art HISR methods. The code is available at https://github.com/Jiahuiqu/S2CycleDiff.

show abstract

“…Dynamic filters networks, which can adjust their structures or weights to different inputs, have been proven to be effective in several tasks [29], [30]. Dai et al [31] propose a deformable convolution to adjust receptive fields according to the learned offsets.…”

Section: B Dynamic Filters Networkmentioning

confidence: 99%

Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion

Wang,

Mao,

Liu

et al. 2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

RGB-guided depth completion aims at predicting dense depth maps from sparse depth measurements and corresponding RGB images, where how to effectively and efficiently exploit the multi-modal information is a key issue. Guided dynamic filters, which generate spatially-variant depth-wise separable convolutional filters from RGB features to guide depth features, have been proven to be effective in this task. However, the dynamically generated filters require massive model parameters, computational costs and memory footprints when the number of feature channels is large. In this paper, we propose to decompose the guided dynamic filters into a spatially-shared component multiplied by content-adaptive adaptors at each spatial location. Based on the proposed idea, we introduce two decomposition schemes A and B, which decompose the filters by splitting the filter structure and using spatial-wise attention, respectively. The decomposed filters not only maintain the favorable properties of guided dynamic filters as being content-dependent and spatiallyvariant, but also reduce model parameters and hardware costs, as the learned adaptors are decoupled with the number of feature channels. Extensive experimental results demonstrate that the methods using our schemes outperform state-of-the-art methods on the KITTI dataset, and rank 1st and 2nd on the KITTI benchmark at the time of submission. Meanwhile, they also achieve comparable performance on the NYUv2 dataset. In addition, our proposed methods are general and could be employed as plug-and-play feature fusion blocks in other multimodal fusion tasks such as RGB-D salient object detection.

show abstract

LAGConv: Local-Context Adaptive Convolution Kernels with Global Harmonic Bias for Pansharpening

Cited by 36 publications

References 31 publications

Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration

Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration

S2CycleDiff: Spatial-Spectral-Bilateral Cycle-Diffusion Framework for Hyperspectral Image Super-resolution

Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion

Contact Info

Product

Resources

About