Spatiotemporal Fusion of Remote Sensing Images with Structural Sparsity and Semi-Coupled Dictionary Learning

Wei, Jingbo; Wang, Lizhe; Liu, Peng; Song, Wenzhe

doi:10.3390/rs9010021

Cited by 36 publications

(22 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this method, the sparse representation is utilized to realize the super-resolution of fine temporal resolution images and a high-pass modulation is applied for fusion. Wei et al [23] included prior knowledge to increase the accuracy of the sparse representation-based method. This method builds a model containing semi-coupled dictionary learning and structural sparsity.…”

Section: Introductionmentioning

confidence: 99%

Quantifying the Effect of Registration Error on Spatio-Temporal Fusion

Tang

Wang

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

It is challenging to acquire satellite sensor data with both fine spatial and fine temporal resolution, especially for monitoring at global scales. Amongst the widely used global monitoring satellite sensors, Landsat data have a coarse temporal resolution, but fine spatial resolution, while MODIS data have fine temporal resolution, but coarse spatial resolution. One solution to this problem is to blend the two types of data using spatio-temporal fusion, creating images with both fine temporal and fine spatial resolution. However, reliable geometric registration of images acquired by different sensors is a prerequisite of spatio-temporal fusion. Due to the potentially large differences between the spatial resolutions of the images to be fused, the geometric registration process always contains some degree of uncertainty. This paper analyzes quantitatively the influence of geometric registration error on spatio-temporal fusion. The relationship between registration error and the accuracy of fusion was investigated under the influence of different temporal distances between images, different spatial patterns within the images and using different methods (i.e., spatial and temporal adaptive reflectance fusion model (STARFM) and Fit-FC; two typical spatio-temporal fusion methods). The results show that registration error has a significant impact on the accuracy of spatio-temporal fusion: as the registration error increased, the accuracy decreased monotonically. The effect of registration error in a heterogeneous region was greater than that in a homogeneous region. Moreover, the accuracy of fusion was not dependent on the temporal distance between images to be fused, but rather on their statistical correlation. Finally, the Fit-FC method was found to be more accurate than the STARFM method, under all registration error scenarios.  Index Terms-Remote sensing data, Landsat, MODIS, spatio-temporal fusion, registration error.

show abstract

Section: Introductionmentioning

confidence: 99%

Quantifying the Effect of Registration Error on Spatio-Temporal Fusion

Tang

Wang

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Then, the one-pair learning method was further developed to apply the SPSTFM method to the case of one known pair of fine and coarse images [27]. Since SPSTFM's assumption that the sparse coefficients across the fine and coarse image patches are the same is too strict, subsequent studies have been devoted to relaxing this assumption, such as the error-bound-regularized sparse coding (EBSPTM) [28], block Sparse Bayesian Learning for Semi-Coupled Dictionary Learning (bSBL-SCDL) [29], and compressed sensing for spatiotemporal fusion (CSSF) [30]. Although these dictionary pair-based methods can predict both the phenological and land-cover changes, the high computational complexity of sparse coding limits their applicability.…”

Section: Introductionmentioning

confidence: 99%

A Novel Deep Learning-Based Spatiotemporal Fusion Method for Combining Satellite Images with Different Resolutions Using a Two-Stream Convolutional Neural Network

et al. 2020

View full text Add to dashboard Cite

Spatiotemporal fusion is considered a feasible and cost-effective way to solve the trade-off between the spatial and temporal resolution of satellite sensors. Recently proposed learning-based spatiotemporal fusion methods can address the prediction of both phenological and land-cover change. In this paper, we propose a novel deep learning-based spatiotemporal data fusion method that uses a two-stream convolutional neural network. The method combines both forward and backward prediction to generate a target fine image, where temporal change-based and a spatial information-based mapping are simultaneously formed, addressing the prediction of both phenological and land-cover changes with better generalization ability and robustness. Comparative experimental results for the test datasets with phenological and land-cover changes verified the effectiveness of our method. Compared to existing learning-based spatiotemporal fusion methods, our method is more effective in predicting phenological change and directly reconstructing the prediction with complete spatial details without the need for auxiliary modulation.

show abstract

“…This was done by introducing a two-layer fusion framework, which first predicted the target image at medium resolution, followed by the desired fine resolution, to tackle the problem of large scale difference between fine and coarse images [7]. In order to tackle the drawback of insufficient prior knowledge for sparse representation such as cluster and joint structural sparsity [16] developed an optimization model for spatio-temporal data fusion using semi-coupled dictionary learning and structural sparsity.…”

Section: Introductionmentioning

confidence: 99%

Spatio-Temporal Data Fusion for Satellite Images Using Hopfield Neural Network

2019

View full text Add to dashboard Cite

Spatio-temporal data fusion refers to the technique of combining high temporal resolution from coarse satellite images and high spatial resolution from fine satellite images. However, data availability remains a major limitation in algorithm development. Existing spatio-temporal data fusion algorithms require at least one known image pair between the fine and coarse resolution image. However, data which come from two different satellite platforms do not necessarily have an overlap in their overpass times, hence restricting the application of spatio-temporal data fusion. In this paper, a new algorithm named Hopfield Neural Network SPatio-tempOral daTa fusion model (HNN-SPOT) is developed by utilizing the optimization concept in the Hopfield neural network (HNN) for spatio-temporal image fusion. The algorithm derives a synthesized fine resolution image from a coarse spatial resolution satellite image (similar to downscaling), with the use of one fine resolution image taken on an arbitrary date and one coarse image taken on a predicted date. The HNN-SPOT particularly addresses the problem when the fine resolution and coarse resolution images are acquired from different satellite overpass times over the same geographic extent. Both simulated datasets and real datasets over Hong Kong and Australia have been used in the evaluation of HNN-SPOT. Results showed that HNN-SPOT was comparable with an existing fusion algorithm, the spatial and temporal adaptive reflectance fusion model (STARFM). HNN-SPOT assumes consistent spatial structure for the target area between the date of data acquisition and the prediction date. Therefore, it is more applicable to geographical areas with little or no land cover change. It is shown that HNN-SPOT can produce accurate fusion results with >90% of correlation coefficient over consistent land covers. For areas that have undergone land cover changes, HNN-SPOT can still produce a prediction about the outlines and the tone of the features, if they are large enough to be recorded in the coarse resolution image at the prediction date. HNN-SPOT provides a relatively new approach in spatio-temporal data fusion, and further improvements can be made by modifying or adding new goals and constraints in its HNN architecture. Owing to its lower demand for data prerequisites, HNN-SPOT is expected to increase the applicability of fine-scale applications in remote sensing, such as environmental modeling and monitoring.

show abstract

Spatiotemporal Fusion of Remote Sensing Images with Structural Sparsity and Semi-Coupled Dictionary Learning

Cited by 36 publications

References 22 publications

Quantifying the Effect of Registration Error on Spatio-Temporal Fusion

Quantifying the Effect of Registration Error on Spatio-Temporal Fusion

A Novel Deep Learning-Based Spatiotemporal Fusion Method for Combining Satellite Images with Different Resolutions Using a Two-Stream Convolutional Neural Network

Spatio-Temporal Data Fusion for Satellite Images Using Hopfield Neural Network

Contact Info

Product

Resources

About