An Enhanced Single-Pair Learning-Based Reflectance Fusion Algorithm with Spatiotemporally Extended Training Samples

Abstract

Spatiotemporal fusion methods are considered a useful tool for generating multi-temporal reflectance data with limited high-resolution images and necessary low-resolution images. In particular, the superiority of sparse representation-based spatiotemporal reflectance fusion model (SPSTFM) in capturing phenology and type changes of land covers has been preliminarily demonstrated. Meanwhile, the dictionary training process, which is a key step in the sparse learning-based fusion algorithm, and its effect on fusion quality are still unclear. In this paper, an enhanced spatiotemporal fusion scheme based on the single-pair SPSTFM algorithm has been proposed through improving the process of dictionary learning, and then evaluated using two actual datasets, with one representing a rural area with phenology changes and the other representing an urban area with land cover type changes. The validated strategy for enhancing the dictionary learning process is divided into two modes to enlarge the training datasets with spatially and temporally extended samples. Compared to the original learning-based algorithm and other employed typical single-pair-based fusion models, experimental results from the proposed fusion method with two extension modes show improved performance in modeling reflectance using the two preceding datasets. Furthermore, the strategy with temporally extended training samples is more effective than the strategy with spatially extended training samples for the land cover area with phenology changes, whereas it is opposite for the land cover area with type changes.

Spatio-temporal fusion for remote sensing data: an overview and new benchmark

“…For instance, the enhanced one-pair learning SPSTFM combines a spatially extended mode and a temporally extended mode to increase the training set. This approach successfully improves the performance of the one-pair learning SPSTFM [59]. The method in [60] exploits high-spectral correlation (across the spectral domain) and high self-similarity (across the spatial domain) to learn a spatio-spectral fusion basis, and then associates temporal changes using a local constraint sparse representation to develop a spatial-spectral-temporal fusion model.…”

STF-EGFA: A Remote Sensing Spatiotemporal Fusion Network with Edge-Guided Feature Attention

“…Chen et al [26] proposed a hierarchical spatiotemporal adaptive fusion model (HSTAFM) that adaptively fuses multisensor features to accurately capture seasonal changes and land use/cover changes by enhancing coarse-resolution images with super-resolution information based on sparse representations, followed by preselection for temporal changes, and selecting similar pixels using a two-level strategy. On this basis, Li et al [27] designed a single-pair learning-based SPSTFM method, combining spatial and temporal expansion models to increase the training set, improve the spatial resolution of high temporal resolution by improving dictionary learning, combine high-pass information obtained by the module fused with high spatial resolution imagery, and successfully improve the accuracy of spatiotemporal prediction. This method improves the prediction effect of SPSTFM.…”

A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks

“…Owing to the fact that learning-based methods are theoretically able to work in any scenario, including cases with significant changes, which are difficult to handle for the weighted function-based and unmixingbased methods, in recent years there has been a significant interest in this kind of approaches. Other than sparse representation-based approaches [16][17][18][19][20][21], techniques such as regression trees [22], random showing the bias between the ground-truth and sensor 1, (k) showing the bias between the ground-truth and sensor 2, and (l) showing the bias between sensors 1 and 2, respectively. It can be seen that the bias between sensors 1 and 2 (i.e., (l)) is significant, which is expected to play an essential role in the STF process.…”

“…After comparing (19) and 20, we can conclude that the STFDCNN (which uses ∆C to approximate ∆F ) can be regarded as a traditional method (i.e., it assumes that change information can be straightforwardly transferred from one sensor to another). Therefore, a comparison between the losses of BiaSTF and STFDCNN can be considered as a comparison between the sensor-bias derived STF model and the traditional model.…”