Multi-Scale Feature Mapping Network for Hyperspectral Image Super-Resolution

Abstract

Hyperspectral Image (HSI) can continuously cover tens or even hundreds of spectral segments for each spatial pixel. Limited by the cost and commercialization requirements of remote sensing satellites, HSIs often lose a lot of information due to insufficient image spatial resolution. For the high-dimensional nature of HSIs and the correlation between the spectra, the existing Super-Resolution (SR) methods for HSIs have the problems of excessive parameter amount and insufficient information complementarity between the spectra. This paper proposes a Multi-Scale Feature Mapping Network (MSFMNet) based on the cascaded residual learning to adaptively learn the prior information of HSIs. MSFMNet simplifies each part of the network into a few simple yet effective network modules. To learn the spatial-spectral characteristics among different spectral segments, a multi-scale feature generation and fusion Multi-Scale Feature Mapping Block (MSFMB) based on wavelet transform and spatial attention mechanism is designed in MSFMNet to learn the spectral features between different spectral segments. To effectively improve the multiplexing rate of multi-level spectral features, a Multi-Level Feature Fusion Block (MLFFB) is designed to fuse the multi-level spectral features. In the image reconstruction stage, an optimized sub-pixel convolution module is used for the up-sampling of different spectral segments. Through a large number of verifications on the three general hyperspectral datasets, the superiority of this method compared with the existing hyperspectral SR methods is proved. In subjective and objective experiments, its experimental performance is better than its competitors.

Hyperspectral Image Super-Resolution Based on Feature Diversity Extraction

“…Following this, we performed an in-depth analysis of the CSSFENet's performance. Finally, we conducted a comparative assessment by pitting the proposed CSSFENet against other methods, including Bicubic, VDSR [27], EDSR [28], MCNet [9], MSDformer, MSFMNet [16], AS 3 ITransUnet, and PDENet [29].…”

“…The method utilizes degradation models and multiscale attentional fusion to improve mapping learning, resulting in superior results with various datasets. Furthermore, Zhang et al [16] proposed a CNN-based super-resolution reconstruction algorithm using multiscale feature extraction and a multilevel feature fusion structure to solve the problem of the lack of effective model design for spectral segment feature learning in hyperspectral remote sensing images. In 2023, Zhang et al [17] proposed the spectral correlation and spatial high-lowfrequency information of a hyperspectral image super-resolution network (SCSFINet) based on spectrum-guided attention for analyzing the information acquired from hyperspectral images.…”

“…The CAVE dataset comprises 32 scenes, each containing 31 spectral segments with a spatial resolution of 512 × 512 pixels, covering the wavelength range of 400-700 nm. During the assessment phase, we designated seven images for testing while reserving 25 for training [16] and the remainder for testing, as shown in Figure 5. The Pavia dataset consists of 102 spectral bands, whereas the PaviaU dataset consists of 103 spectral bands.…”

Hyperspectral image super-resolution reconstruction based on image partition and detail enhancement

“…Li et al [30] proposed a band-attentive adversarial learning frame, which introduces a series of spatial spectral constraints to improve the reconstruction results. To achieve e cient exploitation of high multispectral features, MSFMNet [31] uses a multiscale feature generation, a fusion multiscale feature mapping block based on wavelet transform, and a spatial attention mechanism to learn the spectral features between different spectral bands. These extremely deep networks achieve good recon guration performance.…”

IFE-Net: An Integrated Feature Extraction Network for Single-Image Dehazing

“…Multiscale feature extraction is very effective in the field of dehazing, while maintaining scale invariance and extracting information [42][43][44][45]. Moreover, parallel convolutions with different filter sizes are used to capture features at different scales.…”