A Novel Sharpening Approach for Superresolving Multiresolution Optical Images

Paris, Claudia; Bioucas-Dias, José M.; Bruzzone, Lorenzo

doi:10.1109/tgrs.2018.2867284

Cited by 24 publications

(25 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, the spectral mapping learned in each subspace according to Equation (5) is more reasonable than that learned in the whole spectrum space. If the spectra in can be clustered into the same subspace as Equation (5), Equation (1) can be rewritten as: (6) where and denote the corresponding clustered subspace in cluster , and . As the spectral mapping established in the low-scale spatial resolution is identical to that in the highscale spatial resolution, the target HHS image can be obtained from the clustered HMS image according to Equation (6), by using the trained nonlinear spectral mapping of its corresponding cluster in Equation (5).…”

Section: Problem Formulationmentioning

confidence: 99%

“…Using maximum a posteriori (MAP) estimation, Hardie et al [5] proposed an enhancement method for hyperspectral image using a panchromatic image. In extension, Paris et al [6] proposed a method to enhance the spatial resolution of MS multiresolution images that have more than one high-spatial-resolution channel. Using WorldView-3 data, Selva et al [7] proposed an extended pansharpening method to improve the spatial resolution of hyperspectral image, which introduces the histogram matching operation into the synthesized band variant.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks

Han

Luo

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

Fusion of the high-spatial-resolution hyperspectral (HHS) image using low-spatial- resolution hyperspectral (LHS) and high-spatial-resolution multispectral (HMS) image is usually formulated as a spatial super-resolution problem of LHS image with the help of an HMS image, and that may result in the loss of detailed structural information. Facing the above problem, the fusion of HMS with LHS image is formulated as a nonlinear spectral mapping from an HMS to HHS image with the help of an LHS image, and a novel cluster-based fusion method using multi-branch BP neural networks (named CF-BPNNs) is proposed, to ensure a more reasonable spectral mapping for each cluster. In the training stage, considering the intrinsic characteristics that the spectra are more similar within each cluster than that between clusters and so do the corresponding spectral mapping, an unsupervised clustering is used to divide the spectra of the down-sampled HMS image (marked as LMS) into several clusters according to spectral correlation. Then, the spectrum-pairs from the clustered LMS image and the corresponding LHS image are used to train multi-branch BP neural networks (BPNNs), to establish the nonlinear spectral mapping for each cluster. In the fusion stage, a supervised clustering is used to group the spectra of HMS image into the clusters determined during the training stage, and the final HHS image is reconstructed from the clustered HMS image using the trained multi-branch BPNNs accordingly. Comparison results with the related state-of-the-art methods demonstrate that our proposed method achieves a better fusion quality both in spatial and spectral domains.

show abstract

Section: Problem Formulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks

Han

Luo

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Existing SR on S2 images can be roughly categorized into three classes: pansharpening methods [6][7][8][9][10][11], deep learning methods [12][13][14] and model-based methods [15][16][17][18][19].…”

Section: Introductionmentioning

confidence: 99%

“…Model-based methods [15][16][17][18][19] formulate the superresolution problem as an ill-posed inverse problem and then estimate the high-resolution images under the variational regularization framework by solving an optimization problem that super-resolves all bands simultaneously. These methods rely on an observation model, which generally contains two terms: the fidelity term and the regularization term.…”

Section: Introductionmentioning

confidence: 99%

NSTMR: Super Resolution of Sentinel-2 Images Using Nonlocal Nonconvex Surrogate of Tensor Multirank

Wang

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In this work, we address the super-resolution problems which estimate the high-resolution multispectral images from the multispectral Sentinel-2 (S2) images with different resolutions. Since S2 images can be naturally represented by tensors, we reformulate the degradation processas the tensorbased form. Based on the degradation mechanism, we build a tensor-based optimization model for S2 images super-resolution problem which fully exploitsintrinsicnonlocal spatial similarity and global spectral redundancy. Specifically, the model consists of the data fidelity term and the low-multi-rank regularizer tailored to thoroughly mining the inherent spatial-nonlocal and spectral redundancy. Then we develop an efficientalternating direction method of multipliersalgorithmwith theoretically guaranteed convergenceto tackle the resulting tensor optimization problem. Numerical experiments including simulated and real data demonstrate that our method outperforms the competing methods visually and qualitatively.

show abstract

“…The two techniques called SupReME [22] and S2Sharp [23] solve the problem with a roughness regularizer. While the methods called MuSA [24] and SSSS [25] solve the same problem with a self-similarity regularizer. Those methods can achieve state-of-the-art results, but it is hard to fine-tune the hyper-parameters (e.g., the regularizer weights).…”

Section: Introductionmentioning

confidence: 99%

Sentinel-2 Sharpening Using a Single Unsupervised Convolutional Neural Network With MTF-Based Degradation Model

Nguyen

Ulfarsson

Sveinsson

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The Sentinel-2 (S2) constellation provides multispectral images at 10 m, 20 m, and 60 m resolution bands. Obtaining all bands at 10 m resolution would benefit many applications. Recently, many model-based and deep learning (DL)-based sharpening methods have been proposed. However, the downside of those methods is that the DL-based methods need to be trained separately for the 20 m and the 60 m bands in a supervised manner at reduced-resolution, while the modelbased methods heavily depend on the hand-crafted image priors. To break the gap, this paper proposes a novel unsupervised DLbased S2 sharpening method using a single convolutional neural network (CNN) to sharpen the 20 m and 60 m bands at the same time at full-resolution. The proposed method replaces the handcrafted image prior by the deep image prior (DIP) provided by a CNN structure whose parameters are easily optimized using a DL optimizer. We also incorporate the modulation transfer function (MTF)-based degradation model as a network layer, and add all bands to both network input and output. This setting improves the DIP and exploits the advantage of multitask learning since all S2 bands are highly correlated. Extensive experiments with real S2 data show that our proposed method outperforms competitive methods for reduced-resolution evaluation and yields very high quality sharpened image for full-resolution evaluation.

show abstract

A Novel Sharpening Approach for Superresolving Multiresolution Optical Images

Cited by 24 publications

References 47 publications

Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks

Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks

NSTMR: Super Resolution of Sentinel-2 Images Using Nonlocal Nonconvex Surrogate of Tensor Multirank

Sentinel-2 Sharpening Using a Single Unsupervised Convolutional Neural Network With MTF-Based Degradation Model

Contact Info

Product

Resources

About