An object-based spatiotemporal fusion model for remote sensing images

Hua, Zhang; Sun, Yue; Shi, Wenzhong; Guo, Dizhou; Zheng, Nanning

doi:10.1080/22797254.2021.1879683

Cited by 16 publications

(6 citation statements)

References 60 publications

(64 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Global and local unmixing models were applied to obtain strong spectral information changes while preserving structural information. Object-based spatiotemporal fusion model (OBSTFM) [19] is another recent spatiotemporal fusion technique. It first applies multi-resolution segmentation to separate the various spectral and surface features into different objects, then a linear injection model estimates the fine image via an injection gain.…”

Section: To Enhance Multiple Image Resolutions Techniques Such Asmentioning

confidence: 99%

“…This means that X tot needs to include X i . For the second part, since (X i , Y i ) and (X i−1 , Y i−1 ) are not alike, information of the previously estimated abundance map cannot bring us closer to the optimal solution of the present abundance map, and S i−1 is thus set to 0 in (19). By also setting S mi to 0, S ai can represent w.l.o.g.…”

Section: Fusion Of the Cfst Hsis And The Fsct Msismentioning

confidence: 99%

“…Owing to a lack of hyperspectral STSR peer methods to compare with, we extend the usage of the benchmark multispectral STSR methods, including STARFM [12], Fit-FC [14], SFSDAF [17], RASDF [18], and OBSTFM [19], which are introduced in Section I. X i , i ∈ I 4 and Z 1 are used as inputs for the peer methods since all of the peer methods specialize in estimating FSFT images by a CSFT image set and a fine spatial resolution image with identical spectral resolutions.…”

Section: A Data Acquisitionmentioning

confidence: 99%

See 2 more Smart Citations

Optimization-Based Hyperspectral Spatiotemporal Super-Resolution

Chang

Lin

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Due to hardware limitations and financial considerations, it is challenging to acquire fine spatial and temporal resolution (FSFT) images, which leads to the interest in spatiotemporal fusion problems. Instead of directly obtaining costly FSFT images, an alternative is to fuse fine spatial, coarse temporal resolution (FSCT) images with coarse spatial, fine temporal resolution (CSFT) images. Unlike existing spatiotemporal methods which are only designed for multispectral images, this paper first proposes a new fusion framework for hyperspectral spatiotemporal super-resolution, termed HSTSR. In this paper, we first deal with the coarse temporal resolution issue by adopting the fast iterative shrinkage-thresholding algorithm (FISTA) to estimate the missing images at the intermediate time series. Then, we fuse the hyperspectral image and the multispectral image in each time series via coupled nonnegative matrix factorization (CNMF) to get FSFT hyperspectral images. Importantly, we can automatically estimate the associated spatial blurring and spectral downsampling matrices without prior satellite hardware information. Compared with other extended multispectral spatiotemporal methods, our method not only attains satisfying qualities significantly faster, but also requires much less input data.

show abstract

Section: To Enhance Multiple Image Resolutions Techniques Such Asmentioning

confidence: 99%

Section: Fusion Of the Cfst Hsis And The Fsct Msismentioning

confidence: 99%

Section: A Data Acquisitionmentioning

confidence: 99%

See 1 more Smart Citation

Optimization-Based Hyperspectral Spatiotemporal Super-Resolution

Chang

Lin

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Recently, Zhang et al [59] presented an object-based STIF model with multi-resolution segmentation, linear injection, and spatial filtering. The object extraction and selection of spectrally similar pixels in their approach may be similar to HIFOW.…”

Section: Future Research Directionsmentioning

confidence: 99%

An Object-Based Weighting Approach to Spatiotemporal Fusion of High Spatial Resolution Satellite Images for Small-Scale Cropland Monitoring

Park

2022

Agronomy

View full text Add to dashboard Cite

Continuous crop monitoring often requires a time-series set of satellite images. Since satellite images have a trade-off in spatial and temporal resolution, spatiotemporal image fusion (STIF) has been applied to construct time-series images at a consistent scale. With the increased availability of high spatial resolution images, it is necessary to develop a new STIF model that can effectively reflect the properties of high spatial resolution satellite images for small-scale crop field monitoring. This paper proposes an advanced STIF model using a single image pair, called high spatial resolution image fusion using object-based weighting (HIFOW), for blending high spatial resolution satellite images. The four-step weighted-function approach of HIFOW includes (1) temporal relationship modeling, (2) object extraction using image segmentation, (3) weighting based on object information, and (4) residual correction to quantify temporal variability between the base and prediction dates and also represent both spectral patterns at the prediction date and spatial details of fine-scale images. The specific procedures tailored for blending fine-scale images are the extraction of object-based change and structural information and their application to weight determination. The potential of HIFOW was evaluated from the experiments on agricultural sites using Sentinel-2 and RapidEye images. HIFOW was compared with three existing STIF models, including the spatial and temporal adaptive reflectance fusion model (STARFM), flexible spatiotemporal data fusion (FSDAF), and Fit-FC. Experimental results revealed that the HIFOW prediction could restore detailed spatial patterns within crop fields and clear crop boundaries with less spectral distortion, which was not represented in the prediction results of the other three models. Consequently, HIFOW achieved the best prediction performance in terms of accuracy and structural similarity for all the spectral bands. Other than the reflectance prediction, HIFOW also yielded superior prediction performance for blending normalized difference vegetation index images. These findings indicate that HIFOW could be a potential solution for constructing high spatial resolution time-series images in small-scale croplands.

show abstract

“…In order to overcome the uncertainty in selecting neighboring similar pixels, an improved ESTARFM method was developed based on the landcover endmember type [ 47 ]. To improve the prediction accuracy in non-shape-changing regions, Zhang et al proposed an object-based method [ 48 ]. Aiming to improve the prediction accuracy of ESTARFM, a statistical method was proposed to adaptively determine the window size [ 42 ].…”

Section: Introductionmentioning

confidence: 99%

An Improved Spatiotemporal Data Fusion Method for Snow-Covered Mountain Areas Using Snow Index and Elevation Information

Gao

Liu

et al. 2022

Sensors

View full text Add to dashboard Cite

Remote sensing images with high spatial and temporal resolution in snow-covered areas are important for forecasting avalanches and studying the local weather. However, it is difficult to obtain images with high spatial and temporal resolution by a single sensor due to the limitations of technology and atmospheric conditions. The enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) can fill in the time-series gap of remote sensing images, and it is widely used in spatiotemporal fusion. However, this method cannot accurately predict the change when there is a change in surface types. For example, a snow-covered surface will be revealed as the snow melts, or the surface will be covered with snow as snow falls. These sudden changes in surface type may not be predicted by this method. Thus, this study develops an improved spatiotemporal method ESTARFM (iESTARFM) for the snow-covered mountain areas in Nepal by introducing NDSI and DEM information to simulate the snow-covered change to improve the accuracy of selecting similar pixels. Firstly, the change in snow cover is simulated according to NDSI and DEM. Then, similar pixels are selected according to the change in snow cover. Finally, NDSI is added to calculate the weights to predict the pixels at the target time. Experimental results show that iESTARFM can reduce the bright abnormal patches in the land area compared to ESTARFM. For spectral accuracy, iESTARFM performs better than ESTARFM with the root mean square error (RMSE) being reduced by 0.017, the correlation coefficient (r) being increased by 0.013, and the Structural Similarity Index Measure (SSIM) being increased by 0.013. For spatial accuracy, iESTARFM can generate clearer textures, with Robert’s edge (Edge) being reduced by 0.026. These results indicate that iESTARFM can obtain higher prediction results and maintain more spatial details, which can be used to generate dense time series images for snow-covered mountain areas.

show abstract

An object-based spatiotemporal fusion model for remote sensing images

Cited by 16 publications

References 60 publications

Optimization-Based Hyperspectral Spatiotemporal Super-Resolution

Optimization-Based Hyperspectral Spatiotemporal Super-Resolution

An Object-Based Weighting Approach to Spatiotemporal Fusion of High Spatial Resolution Satellite Images for Small-Scale Cropland Monitoring

An Improved Spatiotemporal Data Fusion Method for Snow-Covered Mountain Areas Using Snow Index and Elevation Information

Contact Info

Product

Resources

About