Sentinel-2 Image Fusion Using a Deep Residual Network

Pálsson, Frosti; Sveinsson, Jóhannes R.; Ulfarsson, Magnus O.

doi:10.3390/rs10081290

Cited by 44 publications

(34 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike the semantic segmentation, where the ground truth can be provided to learn the super parameters in the CNN architectures, the required high-spatial-resolution multispectral imagery is not available. Thus, the important assumption behind the proposed CNN-based pan-sharpening method is that the complex mapping relationship to be learned is identical at a lower-and higher-resolution scale [45]. Before introducing the proposed CNN-based pan-sharpening method, a brief description of the symbols used in this paper is first given to better explain the idea.…”

Section: Proposed Cnn-based Pan-sharpening Methodsmentioning

confidence: 99%

A CNN-Based Pan-Sharpening Method for Integrating Panchromatic and Multispectral Images Using Landsat 8

Cheng

2019

Remote Sensing

View full text Add to dashboard Cite

The increasing availability of sensors enables the combination of a high-spatial-resolution panchromatic image and a low-spatial-resolution multispectral image, which has become a hotspot in recent years for many applications. To address the spectral and spatial distortions that adversely affect the conventional methods, a pan-sharpening method based on a convolutional neural network (CNN) architecture is proposed in this paper, where the low-spatial-resolution multispectral image is upgraded and integrated with the high-spatial-resolution panchromatic image to produce a new multispectral image with high spatial resolution. Based on the pyramid structure of the CNN architecture, the proposed method has high learning capacity to generate more representative and robust hierarchical features for construction tasks. Moreover, the highly nonlinear fusion process can be effectively simulated by stacking several linear filtering layers, which is suitable for learning the complex mapping relationship between a high-spatial-resolution panchromatic and low-spatial-resolution multispectral image. Both qualitative and quantitative experimental analyses were carried out on images captured from a Landsat 8 on-board operational land imager (LOI) sensor to demonstrate the method’s performance. The results regarding the sensitivity analysis of the involved parameters indicate the effects of parameters on the performance of our CNN-based pan-sharpening approach. Additionally, our CNN-based pan-sharpening approach outperforms other existing conventional pan-sharpening methods with a more promising fusion result for different landcovers, with differences in Erreur Relative Globale Adimensionnelle de Synthse (ERGAS), root-mean-squared error (RMSE), and spectral angle mapper (SAM) of 0.69, 0.0021, and 0.81 on average, respectively.

show abstract

Section: Proposed Cnn-based Pan-sharpening Methodsmentioning

confidence: 99%

A CNN-Based Pan-Sharpening Method for Integrating Panchromatic and Multispectral Images Using Landsat 8

Cheng

2019

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…To assess the effectiveness of our proposed method, we take SupReME [27], ResNet [43] and DSen2Net [23] as benchmark methods. Besides, the bicubic interpolation (Bicubic) is used to illustrate the performance of the naive upsampling without considering spectral correlations.…”

Section: Baselines and Quantitative Evaluation Metricsmentioning

confidence: 99%

“…Subsequently, the residual learning and high-pass preprocessing are applied to improve the results [42]. Using the training data with global coverage, a deep residual neural network termed DSen2Net is trained in [23], while [43] focuses on the single image case sharpening via a ResNet. Regardless of the superiority of the CNN-based sharpening methods, their performance still can be improved: (i) Sentinel-2 images have two kinds of LR bands, but most of the existing methods focus on sharpening 20 m bands and ignore the 60 m bands; (ii) the characteristics of LR bands and auxiliary HR bands are obviously different.…”

Section: Introductionmentioning

confidence: 99%

Sentinel-2 Sharpening via Parallel Residual Network

2020

Remote Sensing

View full text Add to dashboard Cite

Sentinel-2 data is of great utility for a wide range of remote sensing applications due to its free access and fine spatial-temporal coverage. However, restricted by the hardware, only four bands of Sentinel-2 images are provided at 10 m resolution, while others are recorded at reduced resolution (i.e., 20 m or 60 m). In this paper, we propose a parallel residual network for Sentinel-2 sharpening termed SPRNet, to obtain the complete data at 10 m resolution. The proposed network aims to learn the mapping between the low-resolution (LR) bands and ideal high-resolution (HR) bands by three steps, including parallel spatial residual learning, spatial feature fusing and spectral feature mapping. First, rather than using the single branch network, the parallel residual learning structure is proposed to extract the spatial features from different resolution bands separately. Second, the spatial feature fusing is aimed to fully fuse the extracted features from each branch and produce the residual image with spatial information. Third, to keep spectral fidelity, the spectral feature mapping is utilized to directly propagate the spectral characteristics of LR bands to target HR bands. Without using extra training data, the proposed network is trained with the lower scale data synthesized from the observed Sentinel-2 data and applied to the original ones. The data at 10 m spatial resolution can be finally obtained by feeding the original 10 m, 20 m and 60 m bands to the trained SPRNet. Extensive experiments conducted on two datasets indicate that the proposed SPRNet obtains good results in the spatial fidelity and the spectral preservation. Compared with the competing approaches, the SPRNet increases the SRE by at least 1.538 dB on 20 m bands and 3.188 dB on 60 m bands while reduces the SAM by at least 0.282 on 20 m bands and 0.162 on 60 m bands.

show abstract

“…Infrared images distinguish targets from background based on differences in thermal radiation. By combining the complementary information of visible and infrared image, it is possible to generate fused images that are more conducive to human decision-making or computer vision tasks, which has been applied to many fields such as the military, target detection, surveillance and so on [4][5][6][7][8][9]. An excellent image fusion algorithm must contain the following conditions.…”

Section: Introductionmentioning

confidence: 99%

An Efficient Method for Infrared and Visual Images Fusion Based on Visual Attention Technique

et al. 2020

View full text Add to dashboard Cite

Infrared and visible image fusion technology provides many benefits for human vision and computer image processing tasks, including enriched useful information and enhanced surveillance capabilities. However, existing fusion algorithms have faced a great challenge to effectively integrate visual features from complex source images. In this paper, we design a novel infrared and visible image fusion algorithm based on visual attention technology, in which a special visual attention system and a feature fusion strategy based on the saliency maps are proposed. Special visual attention system first utilizes the co-occurrence matrix to calculate the image texture complication, which can select a particular modality to compute a saliency map. Moreover, we improved the iterative operator of the original visual attention model (VAM), a fair competition mechanism is designed to ensure that the visual feature in detail regions can be extracted accurately. For the feature fusion strategy, we use the obtained saliency map to combine the visual attention features, and appropriately enhance the tiny features to ensure that the weak targets can be observed. Different from the general fusion algorithm, the proposed algorithm not only preserve the interesting region but also contain rich tiny details, which can improve the visual ability of human and computer. Moreover, experimental results in complicated ambient conditions show that the proposed algorithm in this paper outperforms state-of-the-art algorithms in both qualitative and quantitative evaluations, and this study can extend to the field of other-type image fusion.

show abstract

Sentinel-2 Image Fusion Using a Deep Residual Network

Cited by 44 publications

References 43 publications

A CNN-Based Pan-Sharpening Method for Integrating Panchromatic and Multispectral Images Using Landsat 8

A CNN-Based Pan-Sharpening Method for Integrating Panchromatic and Multispectral Images Using Landsat 8

Sentinel-2 Sharpening via Parallel Residual Network

An Efficient Method for Infrared and Visual Images Fusion Based on Visual Attention Technique

Contact Info

Product

Resources

About