A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions

Fan, Yibo; Wang, Feng; Wang, Haipeng

doi:10.3390/rs14051175

Cited by 19 publications

(10 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The transformation matrix (T) for control point pairs is determined through the leastsquares method, and based on this matrix, the residuals of control point pairs are computed, giving rise to RMSE all and RMSE Loo . The presence of more-accurately matched point pairs translates into a better determination of the parameters in the geometric transformation model, thereby yielding enhanced performance [28]. However, it is important to note that an increase in the number of correctly matched point pairs acquired through a registration method does not necessarily guarantee a decrease in the root-mean-squared error.…”

Section: Quantitative Evaluation Resultsmentioning

confidence: 99%

“…Fang Shang [24] constructed position vectors and change vectors that cleverly characterize image pixels and classified Polarimetric Synthetic Aperture Radar (PolSAR) images of complex terrain by a Quaternion Neural Network (QNN), which is not influenced by height information. Moreover, advanced techniques integrate self-learning with SIFT feature points for near-subpixel-level registration [7], employ deep forest models to enhance robustness [13], utilize unsupervised learning frameworks for multiscale registration [25][26][27], and leverage Transformer networks for efficient and accurate registration [28][29][30][31][32][33]. Deng, X.…”

Section: Deep Learningmentioning

confidence: 99%

“…Fan, Y. [28] introduced an advanced and high-precision dense matching technique, specifically tailored for registering SAR images in situations characterized by weak texture conditions. The approaches of B. Zou [34] and Ming Zhao [35] involve the adoption of a pseudo-label-generation method, eliminating the need for additional annotations.…”

Section: Deep Learningmentioning

confidence: 99%

See 2 more Smart Citations

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

Zhou,

Wang,

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

For Synthetic Aperture Radar (SAR) image registration, successive processes following feature extraction are required by both the traditional feature-based method and the deep learning method. Among these processes, the feature matching process—whose time and space complexity are related to the number of feature points extracted from sensed and reference images, as well as the dimension of feature descriptors—proves to be particularly time consuming. Additionally, the successive processes introduce data sharing and memory occupancy issues, requiring an elaborate design to prevent memory leaks. To address these challenges, this paper introduces the OptionEM-based reinforcement learning framework to achieve end-to-end SAR image registration. This framework outputs registered images directly without requiring feature matching and the calculation of the transformation matrix, leading to significant processing time savings. The Transformer architecture is employed to learn image features, while a correlation network is introduced to learn the correlation and transformation matrix between image pairs. Reinforcement learning, as a decision process, can dynamically correct errors, making it more-efficient and -robust compared to supervised learning mechanisms such as deep learning. We present a hierarchical reinforcement learning framework combined with Episodic Memory to mitigate the inherent problem of invalid exploration in generalized reinforcement learning algorithms. This approach effectively combines coarse and fine registration, further enhancing training efficiency. Experiments conducted on three sets of SAR images, acquired by TerraSAR-X and Sentinel-1A, demonstrated that the proposed method’s average runtime is sub-second, achieving subpixel registration accuracy.

show abstract

Section: Quantitative Evaluation Resultsmentioning

confidence: 99%

Section: Deep Learningmentioning

confidence: 99%

Section: Deep Learningmentioning

confidence: 99%

See 1 more Smart Citation

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

Zhou,

Wang,

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…These studies may not directly address the matching requirements of wide-strip SAR images. To address this issue, Fan et al proposed a precise registration method for large-scale SAR images [ 37 ]. The method utilizes transformers to handle weak textures during image alignment.…”

Section: A Systematic Review Of Transformers In Remote Sensing Image ...mentioning

confidence: 99%

Transformers for Remote Sensing: A Systematic Review and Analysis

Wang,

Ma,

et al. 2024

Sensors

View full text Add to dashboard Cite

Research on transformers in remote sensing (RS), which started to increase after 2021, is facing the problem of a relative lack of review. To understand the trends of transformers in RS, we undertook a quantitative analysis of the major research on transformers over the past two years by dividing the application of transformers into eight domains: land use/land cover (LULC) classification, segmentation, fusion, change detection, object detection, object recognition, registration, and others. Quantitative results show that transformers achieve a higher accuracy in LULC classification and fusion, with more stable performance in segmentation and object detection. Combining the analysis results on LULC classification and segmentation, we have found that transformers need more parameters than convolutional neural networks (CNNs). Additionally, further research is also needed regarding inference speed to improve transformers’ performance. It was determined that the most common application scenes for transformers in our database are urban, farmland, and water bodies. We also found that transformers are employed in the natural sciences such as agriculture and environmental protection rather than the humanities or economics. Finally, this work summarizes the analysis results of transformers in remote sensing obtained during the research process and provides a perspective on future directions of development.

show abstract

“…Synthetic aperture radar (SAR) has the advantages of all-weather, all-day, anti-jamming, far detection, and high concealment [1]. As a remote sensing image data source, SAR images are widely used in scientific research, military reconnaissance, disaster monitoring, resource planning, and natural environment protection [2].…”

Section: Introductionmentioning

confidence: 99%

Ship Detection in SAR Images Based on Feature Enhancement Swin Transformer and Adjacent Feature Fusion

Zhang

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have achieved milestones in object detection of synthetic aperture radar (SAR) images. Recently, vision transformers and their variants have shown great promise in detection tasks. However, ship detection in SAR images remains a substantial challenge because of the characteristics of strong scattering, multi-scale, and complex backgrounds of ship objects in SAR images. This paper proposes an enhancement Swin transformer detection network, named ESTDNet, to complete the ship detection in SAR images to solve the above problems. We adopt the Swin transformer of Cascade-R-CNN (Cascade R-CNN Swin) as a benchmark model in ESTDNet. Based on this, we built two modules in ESTDNet: the feature enhancement Swin transformer (FESwin) module for improving feature extraction capability and the adjacent feature fusion (AFF) module for optimizing feature pyramids. Firstly, the FESwin module is employed as the backbone network, aggregating contextual information about perceptions before and after the Swin transformer model using CNN. It uses single-point channel information interaction as the primary and local spatial information interaction as the secondary for scale fusion based on capturing visual dependence through self-attention, which improves spatial-to-channel feature expression and increases the utilization of ship information from SAR images. Secondly, the AFF module is a weighted selection fusion of each high-level feature in the feature pyramid with its adjacent shallow-level features using learnable adaptive weights, allowing the ship information of SAR images to be focused on the feature maps at more scales and improving the recognition and localization capability for ships in SAR images. Finally, the ablation study conducted on the SSDD dataset validates the effectiveness of the two components proposed in the ESTDNet detector. Moreover, the experiments executed on two public datasets consisting of SSDD and SARShip demonstrate that the ESTDNet detector outperforms the state-of-the-art methods, which provides a new idea for ship detection in SAR images.

show abstract

A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions

Cited by 19 publications

References 45 publications

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

A Sub-Second Method for SAR Image Registration Based on Hierarchical Episodic Control

Transformers for Remote Sensing: A Systematic Review and Analysis

Ship Detection in SAR Images Based on Feature Enhancement Swin Transformer and Adjacent Feature Fusion

Contact Info

Product

Resources

About