Spatiotemporal Fusion of Remote Sensing Images using a Convolutional Neural Network with Attention and Multiscale Mechanisms

Li, Weisheng; Zhang, Xiayan; Peng, Yidong; Dong, Meilin

doi:10.1080/01431161.2020.1809742

Cited by 40 publications

(13 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are some defects in the above method. Deep learning [ 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ] can help to solve this problem. The general idea is: Training two models, model one is used to detect whether the thread roll’s margin is 0; otherwise, we enter model two to detect the thread roll’s margin.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Here, the optimizer selects Adam, which mainly calculates the first-order moment and the second-order moment, and obtains the new step size. For this two-classification problem, BinaryCrossentrop [ 27 , 28 , 29 , 30 , 31 ] is chosen as the loss function. There are only two categories (0,1).…”

Section: Proposed Methodsmentioning

confidence: 99%

“…The main training steps of model two are basically the same as model one. The main difference is that model two is a regression model, and the loss function is Equation (16) [ 27 , 28 , 29 , 30 , 31 ].

is the six-dimensional array marked by the sample, and

is the six-dimensional array output of the model prediction.…”

Section: Proposed Methodsmentioning

confidence: 99%

See 2 more Smart Citations

The Detection of Thread Roll’s Margin Based on Computer Vision

Shi

Wang

2021

Sensors

View full text Add to dashboard Cite

The automatic detection of the thread roll’s margin is one of the kernel problems in the textile field. As the traditional detection method based on the thread’s tension has the disadvantages of high cost and low reliability, this paper proposes a technology that installs a camera on a mobile robot and uses computer vision to detect the thread roll‘s margin. Before starting, we define a thread roll‘s margin as follows: The difference between the thread roll‘s radius and the bobbin’s radius. Firstly, we capture images of the thread roll‘s end surface. Secondly, we obtain the bobbin’s image coordinates by calculating the image’s convolutions with a Circle Gradient Operator. Thirdly, we fit the thread roll and bobbin’s contours into ellipses, and then delete false detections according to the bobbin’s image coordinates. Finally, we restore every sub-image of the thread roll by a perspective transformation method, and establish the conversion relationship between the actual size and pixel size. The difference value of the two concentric circles’ radii is the thread roll’s margin. However, there are false detections and these errors may be more than 19.4 mm when the margin is small. In order to improve the precision and delete false detections, we use deep learning to detect thread roll and bobbin’s radii and then can calculate the thread roll’s margin. After that, we fuse the two results. However, the deep learning method also has some false detections. As such, in order to eliminate the false detections completely, we estimate the thread roll‘s margin according to thread consumption speed. Lastly, we use a Kalman Filter to fuse the measured value and estimated value; the average error is less than 5.7 mm.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Proposed Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The Detection of Thread Roll’s Margin Based on Computer Vision

Shi

Wang

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…(1) Use the 'TensorFlow' and 'Keras' packages in Python to build a CNN framework to construct a remote sensing image glacier segmentation model based on deep learning [73,74]. The glacier segmentation model based on remote sensing images has 24…”

Section: Convolutional Neural Network Classificationmentioning

confidence: 99%

Novel Machine Learning Method Integrating Ensemble Learning and Deep Learning for Mapping Debris-Covered Glaciers

Zhang

Shangguan

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Glaciers in High Mountain Asia (HMA) have a significant impact on human activity. Thus, a detailed and up-to-date inventory of glaciers is crucial, along with monitoring them regularly. The identification of debris-covered glaciers is a fundamental and yet challenging component of research into glacier change and water resources, but it is limited by spectral similarities with surrounding bedrock, snow-affected areas, and mountain-shadowed areas, along with issues related to manual discrimination. Therefore, to use fewer human, material, and financial resources, it is necessary to develop better methods to determine the boundaries of debris-covered glaciers. This study focused on debris-covered glacier mapping using a combination of related technologies such as random forest (RF) and convolutional neural network (CNN) models. The models were tested on Landsat 8 Operational Land Imager (OLI)/Thermal Infrared Sensor (TIRS) data and the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model (ASTER GDEM), selecting Eastern Pamir and Nyainqentanglha as typical glacier areas on the Tibetan Plateau to construct a glacier classification system. The performances of different classifiers were compared, the different classifier construction strategies were optimized, and multiple single-classifier outputs were obtained with slight differences. Using the relationship between the surface area covered by debris and the machine learning model parameters, it was found that the debris coverage directly determined the performance of the machine learning model and mitigated the issues affecting the detection of active and inactive debris-covered glaciers. Various classification models were integrated to ascertain the best model for the classification of glaciers.

show abstract

“…The deep convolutional spatiotemporal fusion network (DCSTFN) [16] uses CNN to extract the main frame and background information from high-resolution images and high-frequency components from low-resolution images [33], and the two extracted features are fused and reconstructed to obtain the prediction results. A convolutional neural network with multiscale and attention mechanisms (AMNet) [34] improved accuracy using a spatial attention mechanism. To further improve the generalization ability and prediction accuracy of the model, Tan proposed an enhanced deep convolutional spatiotemporal fusion network (EDCSFTN) [35], where the relationship between the input and output was obtained entirely by network learning, further improving its accuracy.…”

Section: Introductionmentioning

confidence: 99%

A Spatiotemporal Fusion Method Based on Multiscale Feature Extraction and Spatial Channel Attention Mechanism

et al. 2022

Self Cite

View full text Add to dashboard Cite

Remote sensing satellite images with a high spatial and temporal resolution play a crucial role in Earth science applications. However, due to technology and cost constraints, it is difficult for a single satellite to achieve both a high spatial resolution and high temporal resolution. The spatiotemporal fusion method is a cost-effective solution for generating a dense temporal data resolution with a high spatial resolution. In recent years, spatiotemporal image fusion based on deep learning has received wide attention. In this article, a spatiotemporal fusion method based on multiscale feature extraction and a spatial channel attention mechanism is proposed. Firstly, the method uses a multiscale mechanism to fully utilize the structural features in the images. Then a novel attention mechanism is used to capture both spatial and channel information; finally, the rich features and spatial and channel information are used to fuse the images. Experimental results obtained from two datasets show that the proposed method outperforms existing fusion methods in both subjective and objective evaluations.

show abstract

Spatiotemporal Fusion of Remote Sensing Images using a Convolutional Neural Network with Attention and Multiscale Mechanisms

Cited by 40 publications

References 49 publications

The Detection of Thread Roll’s Margin Based on Computer Vision

The Detection of Thread Roll’s Margin Based on Computer Vision

Novel Machine Learning Method Integrating Ensemble Learning and Deep Learning for Mapping Debris-Covered Glaciers

A Spatiotemporal Fusion Method Based on Multiscale Feature Extraction and Spatial Channel Attention Mechanism

Contact Info

Product

Resources

About