Robust feature matching via neighborhood manifold representation consensus

Ma, Jiayi; Li, Zizhuo; Zhang, Kaining; Shao, Zhenfeng; Xiao, Guobao

doi:10.1016/j.isprsjprs.2021.11.004

Cited by 30 publications

(17 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these deep sparse feature point matching approaches are not able to solve the problem of spatially varying geometric projection problem for image areas with topographic relief. Furthermore, the use of only local appearance information will unavoidably result in a large number of false matches [50], which are not easy to be identified and removed, especially for images with irregular topography [51,52,53].…”

Section: A Deep Learning Based Sparse Feature Point Matching For Opti...mentioning

confidence: 99%

Optical and SAR Image Dense Registration Using a Robust Deep Optical Flow Framework

Zhang

Lei

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The co-registration of optical and SAR imageries is the bottleneck in exploring the complementary information from the two multi-modal datasets. The difficulties lie in not only the complex radiometric relationship between them, but also the distinct geometrical models of the optical and SAR imaging systems, which cause it nontrivial to explicitly depict the spatial relationship between the corresponding image regions when elevation fluctuations exist. This article aims to investigate the optical flow technique for the pixelwise dense registration of the high resolution optical and SAR images, so as to get rid of the outlier removal and geometric mapping procedures, which have to be conducted in the classical image registration approaches that are based on sparse feature point matching. Herein, a deep optical flow framework is designed. Firstly, a dilated feature concatenation method is proposed to enhance the discriminability of the pixelwise features for similarity measurement. An effective network training strategy is used, based on a smoothed flow loss, and also a training dataset that contains simulated elevation fluctuations. Secondly, we propose a self-supervised optical flow finetuning strategy. It incorporates the strength of the blockwise matching approach, which produces better matching precision, into the proposed pixelwise matching procedure. In this way, the accuracy of the optical-SAR dense registration is substantially improved. Extensive experiments conducted on the 1m resolution optical-SAR image pairs of different land-cover types and distinct topographic conditions indicate that, the proposed OSFlowNet-Ft framework is quite robust, and has the potential to perform the optical-SAR image dense registration in practical applications. The Python code of the proposed deep optical flow network will be made available.

show abstract

Section: A Deep Learning Based Sparse Feature Point Matching For Opti...mentioning

confidence: 99%

Optical and SAR Image Dense Registration Using a Robust Deep Optical Flow Framework

Zhang

Lei

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Recently, AI approaches have been considered for many applications of image registration. e proper usage of AI requires the availability of datasets where common training features exist [14][15][16][17][21][22][23]. A survey concerning all image matching techniques is introduced starting with traditional approaches, including Area-Based Matching (ABM) and Feature-Based Matching (ABM) methods, followed by a description of matching methods from handcrafted to deep approaches.…”

Section: Related Workmentioning

confidence: 99%

“…Finally, the registration process should resist any imaging and environmental variations to maximize the similarity among captured scenes. So, the wide scope of registration approaches makes it difficult to compose these approaches since each one concerns a certain problem for a certain application [5,[9][10][11][12][13][14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

SAR Image Matching Based on Local Feature Detection and Description Using Convolutional Neural Network

Elwan¹,

Amein

Mousa

et al. 2022

Security and Communication Networks

View full text Add to dashboard Cite

Feature detection is a vital step for the image registration process whose target is the misalignment correction among images to increase the convergency level. Deep learning (DL) in remote sensing has become a worldwide sensation. Despite its huge potential, DL has not reached its intended target concerning the applications of Synthetic Aperture Radar (SAR) images. In this study, we focus on matching SAR images using a Convolutional Neural Network. The big challenge in this study is how to modify a pretrained Visual Geometry Group model based on the multispectral dataset to act as a SAR image feature detector where it does not require any prior knowledge about the nature of the SAR feature. Since SAR images have different characteristics from optical images such as SAR dynamic range and imaging geometry, some problems arise and should be considered during the matching process. Despite all these difficulties, results demonstrate the robustness of the registration process where it can provide descriptors that preserve the localization data of features. Also, the proposed approach provides reasonable results compared to the state-of-the-art methods and outperforms the correlation approach and ORB descriptor under scaling. In addition, it may be considered an end-to-end image matching tool for SAR images, although the calculations of fine matching parameters are included.

show abstract

“…The conventional matching methods are based on the handcrafted local feature descriptors [4]- [7] to make the representation of two matched features as similar as possible and as discriminant as possible from that of unmatched ones. Over the recent years, the deep learning-based methods have achieved significant progress in general visual tasks, and have also been introduced into the field of image matching [8], [9].…”

Section: Introductionmentioning

confidence: 99%

S2-Net: Self-supervision Guided Feature Representation Learning for Cross-Modality Images

Mei¹

2022

Preprint

View full text Add to dashboard Cite

Combining the respective advantages of crossmodality images can compensate for the lack of information in the single modality, which has attracted increasing attention of researchers into multi-modal image matching tasks. Meanwhile, due to the great appearance differences between cross-modality image pairs, it often fails to make the feature representations of correspondences as close as possible. In this letter, we design a cross-modality feature representation learning network, S2-Net, which is based on the recently successful detect-and-describe pipeline, originally proposed for visible images but adapted to work with cross-modality image pairs. To solve the consequent problem of optimization difficulties, we introduce self-supervised learning with a well-designed loss function to guide the training without discarding the original advantages. This novel strategy simulates image pairs in the same modality, which is also a useful guide for the training of cross-modality images. Notably, it does not require additional data but significantly improves the performance and is even workable for all methods of the detectand-describe pipeline. Extensive experiments are conducted to evaluate the performance of the strategy we proposed, compared to both handcrafted and deep learning-based methods. Results show that our elegant formulation of combined optimization of supervised and self-supervised learning outperforms state-of-thearts on RoadScene and RGB-NIR datasets.

show abstract

Robust feature matching via neighborhood manifold representation consensus

Cited by 30 publications

References 47 publications

Optical and SAR Image Dense Registration Using a Robust Deep Optical Flow Framework

Optical and SAR Image Dense Registration Using a Robust Deep Optical Flow Framework

SAR Image Matching Based on Local Feature Detection and Description Using Convolutional Neural Network

S2-Net: Self-supervision Guided Feature Representation Learning for Cross-Modality Images

Contact Info

Product

Resources

About