A Cross-Resolution, Spatiotemporal Geostatistical Fusion Model for Combining Satellite Image Time-Series of Different Spatial and Temporal Resolutions

Kim, Yeseul; Kyriakidis, Phaedon C.; Park, No-Wook

doi:10.3390/rs12101553

Cited by 6 publications

(6 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We combined the MCDNet with two weight function-based methods STARFM [18], ESTARFM [9], FSDAF [22] based on the unmixing-based method and three deep learning methods based on the learning-based method: stfNet [35], DCSTFN [27], and EDCSTFN [34] for comparison. The purpose is to verify the reconstruction effect of the MCDNet.…”

Section: Comparison and Evaluationmentioning

confidence: 99%

“…Studying different land cover types [1], monitoring seasonal vegetation growth and wilting changes [2], modeling carbon sequestration [3], predicting agricultural plant output [4], monitoring human-made landscapes [5], monitoring atmospheric ecosystem changes [6], monitoring disasters [7], [8] and other field applications. However, because of hardware technology and cost constraints, it is difficult for a single satellite to directly obtain dense time satellite data images with high spatial resolution [9], [10]. Some satellites have a shorter revisit period at the expense of the spatial resolution of the acquired data.…”

Section: Introductionmentioning

confidence: 99%

“…Hilker et al [19] proposed a spatiotemporal adaptive algorithm that considers unknown disturbance events or changes in land cover types. Zhu et al [9] developed the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) that can enhance the effect of heterogeneous landscape prediction, and Shen et al [20] improved the weighting strategy, considering the differences in sensor observations of different land cover types.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Multi-Cooperative Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Li¹,

Yang²,

Peng³

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing satellite images with high temporal and high spatial resolution play a critical role in earth science applications. However, it is difficult for a single satellite to obtain such images due to technical and cost constraints. Therefore, spatiotemporal image fusion based on deep learning has received extensive attention in recent years. This paper proposes a multicooperative deep convolutional neural network (MCDNet) for spatiotemporal satellite image fusion. This method is a new multinetwork model in which multiple networks work together to reconstruct the predicted image. The multinetwork model consists of a super-resolution network, a difference reconstruction network, and a collaborative training network. First, the superresolution network uses the combination of a novel multiscale mechanism and dilated convolutions to make full use of the spectral information of the coarse image and upgrade it to a transitional image that matches the fine image. The difference reconstruction network uses structural relevance to complete the reconstruction of the fine difference image. The collaborative training network extracts the hidden information from the fine image and uses the time relevance to restrict the training of the difference reconstruction network. Finally, the fine difference image and the known fine image are combined to complete the image fusion. The new compound loss function can help multinetwork models better complete cooperative training.Through experiments on two datasets and comparison with existing fusion algorithms, the subjective and objective results prove that MCDNet can effectively reconstruct higher-quality prediction images.

show abstract

Section: Comparison and Evaluationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-Cooperative Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Li¹,

Yang²,

Peng³

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Satellite images with medium or low spatial resolution, such as MODIS and Landsat images, can be effectively utilized for nationwide or regional crop monitoring and thematic mapping [7][8][9]. However, their spatial resolutions are too coarse to be applied for detailed local analysis in small-scale croplands [10]. For example, the average areas of paddy rice fields and dry fields in Korea are 0.14 ha and 0.11 ha, respectively [11].…”

Section: Introductionmentioning

confidence: 99%

An Object-Based Weighting Approach to Spatiotemporal Fusion of High Spatial Resolution Satellite Images for Small-Scale Cropland Monitoring

Park

2022

Agronomy

Self Cite

View full text Add to dashboard Cite

Continuous crop monitoring often requires a time-series set of satellite images. Since satellite images have a trade-off in spatial and temporal resolution, spatiotemporal image fusion (STIF) has been applied to construct time-series images at a consistent scale. With the increased availability of high spatial resolution images, it is necessary to develop a new STIF model that can effectively reflect the properties of high spatial resolution satellite images for small-scale crop field monitoring. This paper proposes an advanced STIF model using a single image pair, called high spatial resolution image fusion using object-based weighting (HIFOW), for blending high spatial resolution satellite images. The four-step weighted-function approach of HIFOW includes (1) temporal relationship modeling, (2) object extraction using image segmentation, (3) weighting based on object information, and (4) residual correction to quantify temporal variability between the base and prediction dates and also represent both spectral patterns at the prediction date and spatial details of fine-scale images. The specific procedures tailored for blending fine-scale images are the extraction of object-based change and structural information and their application to weight determination. The potential of HIFOW was evaluated from the experiments on agricultural sites using Sentinel-2 and RapidEye images. HIFOW was compared with three existing STIF models, including the spatial and temporal adaptive reflectance fusion model (STARFM), flexible spatiotemporal data fusion (FSDAF), and Fit-FC. Experimental results revealed that the HIFOW prediction could restore detailed spatial patterns within crop fields and clear crop boundaries with less spectral distortion, which was not represented in the prediction results of the other three models. Consequently, HIFOW achieved the best prediction performance in terms of accuracy and structural similarity for all the spectral bands. Other than the reflectance prediction, HIFOW also yielded superior prediction performance for blending normalized difference vegetation index images. These findings indicate that HIFOW could be a potential solution for constructing high spatial resolution time-series images in small-scale croplands.

show abstract

“…The weight function-based method is currently the most widely used method, and the spatial and temporal adaptive reflectance fusion model (STARFM) is the most concerned and earliest method to solve the fusion problem by using a weight function [18]. The enhanced STARFM based on STARFM (ES-TARFM) can effectively improve the reconstruction effect of the heterogeneous landscape [19].…”

mentioning

confidence: 99%

A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Yang

Peng

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Due to technology and cost limitations, it is challenging to obtain high temporal and spatial resolution images from a single satellite spectrometer, which significantly limits the specific application of such remote sensing images in earth science. To solve the problem that the existing algorithms cannot effectively balance the spatial detail preservation and spectral change reconstruction, a pseudo-Siamese deep convolutional neural network (PDCNN) for spatiotemporal fusion is proposed in this article. The method proposes a pseudo-Siamese network framework model for fusion. This framework has two independent and equal feature extraction streams, but the weights are not shared. The two feature extraction streams process the image information at the previous and later moments and reconstruct the fine image of the corresponding time to fully extract the image information at different times. In the feature extraction stream, the multiscale mechanism and dilated convolution of flexible perception are designed, which can flexibly obtain feature image information and improve the model reconstruction accuracy. In addition, an attention mechanism is introduced to improve the weight of the crucial information for the remote sensing images. Adding a residual connection enhances the reuse of the initial feature information in shallow networks and reduces the loss of feature information in deep networks. Finally, the fine images obtained from the two feature extraction streams are weighted and fused to obtain the final predicted image. The subjective and objective results demonstrate that the PDCNN can effectively reconstruct the fusion image with higher quality.

show abstract

A Cross-Resolution, Spatiotemporal Geostatistical Fusion Model for Combining Satellite Image Time-Series of Different Spatial and Temporal Resolutions

Cited by 6 publications

References 39 publications

A Multi-Cooperative Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

A Multi-Cooperative Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

An Object-Based Weighting Approach to Spatiotemporal Fusion of High Spatial Resolution Satellite Images for Small-Scale Cropland Monitoring

A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Contact Info

Product

Resources

About