Pansharpening Using Unsupervised Generative Adversarial Networks With Recursive Mixed-Scale Feature Fusion

Abstract

Panchromatic sharpening (pansharpening) is an important technology for improving the spatial resolution of multispectral (MS) images. The majority of the models are implemented at the reduced resolution, leading to unfavorable results at the full resolution. Moreover, the complicated relationship between MS and panchromatic (PAN) images is often ignored in detail injection. For the mentioned problems, unsupervised generative adversarial networks with recursive mixed-scale feature fusion for pansharpening (RMFF-UPGAN) are modeled to boost the spatial resolution and preserve the spectral information. RMFF-UPGAN comprises a generator and two U-shaped discriminators. A dual-stream trapezoidal branch is designed in the generator to obtain multiscale information. Further, a recursive mixed-scale feature fusion subnetwork is designed. Perform a prior fusion on the extracted MS and PAN features of the same scale. A mixed-scale fusion is conducted on the prior fusion results of the fine-scale and coarse-scale. The fusion is executed sequentially in the above manner building a recursive mixed-scale fusion structure and finally generating key information. A compensation information mechanism is also designed for the reconstruction of key information to compensate for information. A nonlinear rectification block for the reconstructed information is developed to overcome the distortion induced by neglecting the complicated relationship between MS and PAN images. Two U-shaped discriminators are designed and a new composite loss function is defined. The presented model is validated using two satellite data and the outcomes reveal better than the prevalent approaches regarding both visual assessment and objective indicators.

MDDCMA: A Distributed Image Fusion Framework Based on Multiscale Dense Dilated Convolution and Coordinate Mean Attention

“…Each node generates an optimal estimate given its data and sends it to the fusion node, which subsequently combines the local estimates to achieve the best centralized estimate. Wu et al [23] proposed a multi-branch image fusion network for multi-focus image fusion. Similarly, Wu et al [35] proposed a distributed fusion framework specifically for low-resolution multispectral images and panchromatic images.…”

“…For instance, Liu et al [21] proposed a CNN-based method for infrared and visible image fusion, where a pre-trained CNN is used to generate fusion weights, and the laplacian pyramid method is used for image fusion. There is another type of method involves the use of CNN [18], [22], [23]. Guided by meticulously designed loss functions and network structures, these approaches achieve end-to-end feature extraction, feature fusion, and image reconstruction.…”

An Intelligent Sensor Data Preprocessing Method for OCT Fundus Image Watermarking Using an RCNN

“…Jiang et al addressed the reliability concerns of zeroing neural networks using a robust neural dynamic model [8]. Wu et al developed an unsupervised generative adversarial network to effectively fuse panchromatic and multispectral images [9]. Recently, deep learning technology has attracted much attention in medical image segmentation owing to its success in vision applications [10][11][12][13].…”

Bounded Adaptive Function Activated Recurrent Neural Network for Solving the Dynamic QR Factorization

“…In [32], the varying-parameter ZNNs are shown to be better than the traditional ZNN model in solving the dynamic Lyapunov function and Stein matrix equation. In practical applications, considering the variations in algorithm parameter values in different domains, some researchers have proposed intelligent optimization algorithms incorporating adaptive coefficients [33][34][35][36]. Chen et al [37] add adaptive parameters to the controller design to update data adaptively and ensure the controller's stability.…”