A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors

Abstract

Abstract-The development of multisensor systems in recent years has led to great increase in the amount of available remote sensing data. Image fusion techniques aim at inferring high quality images of a given area from degraded versions of the same area obtained by multiple sensors. This paper focuses on pansharpening, which is the inference of a high spatial resolution multispectral image from two degraded versions with complementary spectral and spatial resolution characteristics: 1) a low spatial resolution multispectral image and 2) a high spatial resolution panchromatic image. We introduce a new variational model based on spatial and spectral sparsity priors for the fusion. In the spectral domain, we encourage low-rank structure, whereas in the spatial domain, we promote sparsity on the local differences. Given the fact that both panchromatic and multispectral images are integrations of the underlying continuous spectra using different channel responses, we propose to exploit appropriate regularizations based on both spatial and spectral links between panchromatic and fused multispectral images. A weighted version of the vector total variation norm of the data matrix is employed to align the spatial information of the fused image with that of the panchromatic image. With regard to spectral information, two different types of regularization are proposed to promote a soft constraint on the linear dependence between the panchromatic and fused multispectral images. The first one estimates directly the linear coefficients from the observed panchromatic and low-resolution multispectral images by linear regression while the second one employs the principal component pursuit to obtain a robust recovery of the underlying low-rank structure. We also show that the two regularizers are strongly related. The basic idea of both regularizers is that the J. Chanussot is with the GIPSA-Laboratory, University of Grenoble, Grenoble F-38402, France, and also with the Faculty of Electrical and Computer Engineering, University of Iceland, Reykjavik 101, Iceland (e-mail: jocelyn.chanussot@gipsa-lab.grenoble-inp.fr).This paper has supplementary downloadable material available at http://ieeexplore.ieee.org., provided by the author. The supplementary material contains the details of the parameter selection and the algorithm performance of the two proposed methods. It can also be downloaded at http://www.gipsalab.fr/∼jocelyn.chanussot/SupplementaryMaterial.pdf. The total size of the file is 82.5 kB. Contact greenhxy@gmail.com for further questions about this work.Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee. Index Terms-Image fusion, pansharpening, remote sensing, principal component pursuit, total variation, low rank recovery, convex optimization, proximal splitting method, split augmented Lagrangian shrinkage (SALSA).

Multimodal Classification of Remote Sensing Images: A Review and Future Directions

“…A wide variety of image fusion methods have been proposed in the remote sensing literature [7], [17], [18], [38], [39], being pan-sharpening methods [24], [32], [35], [40] among the most widespread. Pan-sharpening uses the PAN band to increase the spatial resolution of the MS bands through image processing [34].…”

Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation

“…However, while HS sensors provide abundant spectral information, their spatial resolution is generally more limited [8], [9]. To obtain images with good spectral and spatial resolutions, the remote sensing community has been devoting increasing research efforts to the problem of fusing HS with MS or PAN images [10]- [13]. From an application point of view, this problem is also important as motivated by recent national programs, e.g., the Japanese next-generation space-borne hyperspectral image suite (HISUI), which fuses co-registered MS and HS images acquired over the same scene under the same conditions [14].…”

Enhanced Dictionary based Sparse Representation Fusion for Multi-temporal Remote Sensing Images

“…And online dictionary learning was used to optimize the problem. He et al [2014] applied regularization control based on the spatial and spectral characteristics of panchromatic and multispectral images recorded by different sensors deployed in the same area or scene. The method also uses both spatial and spectral sparsity priors for image fusion.…”