Fast and Accurate Spatiotemporal Fusion Based Upon Extreme Learning Machine

Liu, Xun; Deng, Chenwei; Wang, Shuigen; Huang, Guang-Bin; Zhao, Baojun; Lauren, Paula

doi:10.1109/lgrs.2016.2622726

Cited by 70 publications

(29 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…forests [23] and extreme learning machines [24] have been widely used. Recently, convolutional neural networks (CNNs) have been widely used in image processing tasks, such as super-resolution [25,26] or pansharpening [27,28], obtaining remarkable results.…”

Section: Y F Et Al Sci China Inf Scimentioning

confidence: 99%

A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks

et al. 2020

Sci. China Inf. Sci.

View full text Add to dashboard Cite

Owing to the tradeoff between scanning swath and pixel size, currently no satellite Earth observation sensors are able to collect images with high spatial and temporal resolution simultaneously. This limits the application of satellite images in many fields, including the characterization of crop yields or the detailed investigation of human-nature interactions. Spatio-temporal fusion (STF) is a widely used approach to solve the aforementioned problem. Traditional STF methods reconstruct fine-resolution images under the assumption that changes are able to be transferred directly from one sensor to another. However, this assumption may not hold in real scenarios, owing to the different capacity of available sensors to characterize changes. In this paper, we model such differences as a bias, and introduce a new sensor bias-driven STF model (called BiaSTF) to mitigate the differences between the spectral and spatial distortions presented in traditional methods. In addition, we propose a new learning method based on convolutional neural networks (CNNs) to efficiently obtain this bias. An experimental evaluation on two public datasets suggests that our newly developed method achieves excellent performance when compared to other available approaches.

show abstract

Section: Y F Et Al Sci China Inf Scimentioning

confidence: 99%

A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks

et al. 2020

Sci. China Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…The Sparse Representation-Based Spatiotemporal Reflectance Fusion Model (SPSTFM) is firstly introduced for fusing two observed image pairs (Landsat and MODIS) [31] and then developed with single image pair [32] for a wide application extension. From the view of computation complex and performance for large image patches, an Extreme Learning Machine (ELM) with rich local structural information is introduced to model learning-based spatiotemporal fusion by learning a mapping function on difference images that is also adopted in SPSTFM [33]. Training and learning steps are known to be key for learning-based fusion methods.…”

Section: Of 16mentioning

confidence: 99%

A Learning-Enhanced Two-Pair Spatiotemporal Reflectance Fusion Model for GF-2 and GF-1 WFV Satellite Data

Chen

et al. 2020

Sensors

View full text Add to dashboard Cite

Since requirements of related applications for time series remotely-sensed images with high spatial resolution have been hard to be satisfied under current observation conditions of satellite sensors, it is key to reconstruct high-resolution images at specified dates. As an effective data reconstruction technique, spatiotemporal fusion can be used to generate time series land surface parameters with a clear geophysical significance. In this study, an improved fusion model based on the Sparse Representation-Based Spatiotemporal Reflectance Fusion Model (SPSTFM) is developed and assessed with reflectance data from Gaofen-2 Multi-Spectral (GF-2 MS) and Gaofen-1 Wide-Field-View (GF-1 WFV). By introducing a spatially enhanced training method to dictionary training and sparse coding processes, the developed fusion framework is expected to promote the description of high-resolution and low-resolution overcomplete dictionaries. Assessment indices including Average Absolute Deviation (AAD), Root-Mean-Square Error (RMSE), Peak Signal to Noise Ratio (PSNR), Correlation Coefficient (CC), spectral angle mapper (SAM), structure similarity (SSIM) and Erreur Relative Global Adimensionnelle de Synthèse (ERGAS) are then used to test employed fusion methods for a parallel comparison. The experimental results show that more accurate prediction of GF-2 MS reflectance than that from the SPSTFM can be obtained and furthermore comparable with popular two-pair based reflectance fusion models like the Spatial and Temporal Adaptive Fusion Model (STARFM) and the Enhanced-STARFM (ESTARFM).

show abstract

“…Although these dictionary pair-based methods can predict both the phenological and land-cover changes, the high computational complexity of sparse coding limits their applicability. To reduce this complexity, the extreme learning machine (ELM), a fast single hidden layer feed-forward neural network, was utilized to learn the nonlinear mapping between the fine and coarse images [31]. Motivated by the advantages of deep nonlinear mapping learning, some relevant spatiotemporal fusion methods have been proposed [32,33].…”

Section: Introductionmentioning

confidence: 99%

A Novel Deep Learning-Based Spatiotemporal Fusion Method for Combining Satellite Images with Different Resolutions Using a Two-Stream Convolutional Neural Network

et al. 2020

View full text Add to dashboard Cite

Spatiotemporal fusion is considered a feasible and cost-effective way to solve the trade-off between the spatial and temporal resolution of satellite sensors. Recently proposed learning-based spatiotemporal fusion methods can address the prediction of both phenological and land-cover change. In this paper, we propose a novel deep learning-based spatiotemporal data fusion method that uses a two-stream convolutional neural network. The method combines both forward and backward prediction to generate a target fine image, where temporal change-based and a spatial information-based mapping are simultaneously formed, addressing the prediction of both phenological and land-cover changes with better generalization ability and robustness. Comparative experimental results for the test datasets with phenological and land-cover changes verified the effectiveness of our method. Compared to existing learning-based spatiotemporal fusion methods, our method is more effective in predicting phenological change and directly reconstructing the prediction with complete spatial details without the need for auxiliary modulation.

show abstract

Fast and Accurate Spatiotemporal Fusion Based Upon Extreme Learning Machine

Cited by 70 publications

References 18 publications

A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks

A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks

A Learning-Enhanced Two-Pair Spatiotemporal Reflectance Fusion Model for GF-2 and GF-1 WFV Satellite Data

A Novel Deep Learning-Based Spatiotemporal Fusion Method for Combining Satellite Images with Different Resolutions Using a Two-Stream Convolutional Neural Network

Contact Info

Product

Resources

About