MSDRN: Pansharpening of Multispectral Images via Multi-Scale Deep Residual Network

Wang, Wen Qing; Zhou, Zhiqiang; Liu, Ran; Xie, Guo

doi:10.3390/rs13061200

Cited by 24 publications

(13 citation statements)

References 48 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…p, q are typically set to 1 and α, β are the tradeoff coefficients, usually α = β = 1 [31]. UQI is the universal image quality indices defined as Equation (15), where σ xy is the covariance of x and y, x, y are the average of x and y, and σ x 2 , σ y 2 are the variance of x and y, respectively. From Equations ( 12)-( 15), it can be concluded that D λ measures the distance of the band correlation between fusion result and MS image.…”

Section: No-reference Lossmentioning

confidence: 99%

“…Due to the continuous combination of the extracted high-level image features and low-level features, the residual network alleviates the problem of image details loss with the deepening of the CNN. However, by constantly feeding the shallow features into the deep network, the number of network parameters and the quantities of features to be processed for the deep layer of the network will increase, which will make the network more complicated [13][14][15]. Subsequently, some methods [16] are proposed to improve the running efficiency of the residual network in the field of computer vision.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pan-Sharpening Based on CNN+ Pyramid Transformer by Using No-Reference Loss

Guo

Liu

2022

Remote Sensing

View full text Add to dashboard Cite

The majority of existing deep learning pan-sharpening methods often use simulated degraded reference data due to the missing of real fusion labels which affects the fusion performance. The normally used convolutional neural network (CNN) can only extract the local detail information well which may cause the loss of important global contextual characteristics with long-range dependencies in fusion. To address these issues and to fuse spatial and spectral information with high quality information from the original panchromatic (PAN) and multispectral (MS) images, this paper presents a novel pan-sharpening method by designing the CNN+ pyramid Transformer network with no-reference loss (CPT-noRef). Specifically, the Transformer is used as the main architecture for fusion to supply the global features, the local features in shallow CNN are combined, and the multi-scale features from the pyramid structure adding to the Transformer encoder are learned simultaneously. Our loss function directly learns the spatial information extracted from the PAN image and the spectral information from the MS image which is suitable for the theory of pan-sharpening and makes the network control the spatial and spectral loss simultaneously. Both training and test processes are based on real data, so the simulated degraded reference data is no longer needed, which is quite different from most existing deep learning fusion methods. The proposed CPT-noRef network can effectively solve the huge amount of data required by the Transformer network and extract abundant image features for fusion. In order to assess the effectiveness and universality of the fusion model, we have trained and evaluated the model on the experimental data of WorldView-2(WV-2) and Gaofen-1(GF-1) and compared it with other typical deep learning pan-sharpening methods from both the subjective visual effect and the objective index evaluation. The results show that the proposed CPT-noRef network offers superior performance in both qualitative and quantitative evaluations compared with existing state-of-the-art methods. In addition, our method has the strongest generalization capability by testing the Pleiades and WV-2 images on the network trained by GF-1 data. The no-reference loss function proposed in this paper can greatly enhance the spatial and spectral information of the fusion image with good performance and robustness.

show abstract

Section: No-reference Lossmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Pan-Sharpening Based on CNN+ Pyramid Transformer by Using No-Reference Loss

Guo

Liu

2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…e structure characteristics of several common residual neural networks are shown as follows [16][17][18] 4…”

Section: Basis Of Residual Neural Networkmentioning

confidence: 99%

Leisure Sports Behavior Recognition Algorithm Based on Deep Residual Network

Chou

2022

Scientific Programming

View full text Add to dashboard Cite

Aiming at the related problems existing in the field of leisure sports computing, in order to study the behavior recognition of leisure sports by deep residual network, based on the deep residual neural network theory, the behavior recognition algorithm and the corresponding robust model are used to analyze the leisure sports related samples, and the correlation model is used to predict and analyze the leisure sports related content. The results show that the change curves of Sig and Tanh functions can be divided into slow increasing stage, linear increasing stage, and stable stage. The y value corresponding to ReLU curve shows a linear change trend with the increase of x value. The Leaky function’s corresponding curve can be divided into two stages. The function coincides with the ReLU function in the first quadrant and remains linear in the third quadrant. The activation function curves corresponding to layers 56 and 20 have a relatively large variation range, and both of them show an overall trend of gradual decline. On the whole, the curve value corresponding to layer 56 is higher than that corresponding to layer 20, indicating that the method of layer 20 is relatively good and the corresponding training error is relatively low. It can be seen from the robustness recognition rate of various methods under different training samples that Fl has the highest overall data recognition rate while Sc has relatively poor stability. However, the recognition rate of IDCC and DCC shows a relatively flat trend, indicating that these two methods have certain advantages in describing the robust recognition rate. The research results can provide theoretical support for the application of deep residual neural networks in other fields.

show abstract

“…Wang W., et al [7] designed a multi-scale deep residual network (MSDRN) for pan-sharpening the MS images. Here, multi-level network was employed for scaling the source images.…”

Section: Neural Network-based Techniquesmentioning

confidence: 99%

Study and Analysis of Various Pan-Sharpening Techniques: A Challenging Overview

Singh¹

2022

View full text Add to dashboard Cite

Satellite sensors acquire massive data to exploit and address numerous human tasks. Due to the physical constraints, the usage of high spatial resolution multispectral (MS) images cannot be acquired by the unique devices. Hence, it is necessary to develop techniques by incorporating the images of MS and the panchromatic (PAN), also called Pan sharpening in remote sensing applications. Hence, this survey analyzes 30 research existing papers in Pan sharpening, which is based on various techniques, like neural network-based techniques, image fusion-based techniques, guided filterbased techniques, and optimization-based techniques. The analysis is based on the publication year of the research paper, research techniques employed, performance evaluation measures, and achievement of the research techniques. Also, the issues in the approaches are explained. Moreover, the future scope is to develop an efficient Pan sharpening technique based on the limitations recognized from the existing research Pan sharpening techniques.

show abstract

MSDRN: Pansharpening of Multispectral Images via Multi-Scale Deep Residual Network

Cited by 24 publications

References 48 publications

Pan-Sharpening Based on CNN+ Pyramid Transformer by Using No-Reference Loss

Pan-Sharpening Based on CNN+ Pyramid Transformer by Using No-Reference Loss

Leisure Sports Behavior Recognition Algorithm Based on Deep Residual Network

Study and Analysis of Various Pan-Sharpening Techniques: A Challenging Overview

Contact Info

Product

Resources

About