ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning

Deng, Yuxin; Ma, Jiayi

doi:10.1109/tip.2022.3231135

Cited by 21 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) Performance comparison to other methods: The performance of the proposed methods is quantitatively and qualitatively compared with some popular multimodal image registration methods, including RIFT [8], ASS [9], sRIFD [43], Pix2pix [44], pix2pixHD [45], KCG-GAN [18], and ReDFeat [46]. Ten pairs are picked at random, and table II lists the NCM results for eight algorithms.…”

Section: Resultsmentioning

confidence: 99%

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Han,

Lv,

Wang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Feature representation is a crucial issue in multimodal image registration. The handcrafted features extracted by traditional methods are highly sensitive to nonlinear radiation differences, while supervised learning methods are limited by deficient labeled samples in the remote sensing field. Therefore, this paper proposes a consistency feature representation learning method for multimodal image registration, which involves mapping data into a common feature space to realize the accurate alignment of remote sensing images. First, a contrastive network with a spatial attention mechanism is driven to enhance the capability to highlight high-level features of images. Second, a positive sample augmentation strategy is implemented with contrastive loss, which helps the model learn the inherent features better, and imposes constraints on the sample similarity to optimize the feature projection. Finally, a multimodal image registration framework is proposed to enhance the stability of feature matching. The proposed framework achieves accurate feature extraction and consistency feature description for multimodal images, ensuring robustness against nonlinear radiometric differences. Experimental results demonstrate that the proposed method obtains more reliable registration results on the SEN1-2 dataset. Furthermore, the proposed algorithm achieves superior performance on data from other modalities, indicating strong generalization ability.

show abstract

Section: Resultsmentioning

confidence: 99%

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Han,

Lv,

Wang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…DescNet [28] performs a multilevel convolutional pooling operation on the input image block to obtain the deep features of the image block and finally outputs a 128-dimensional vector as a feature descriptor. ReDFeat [29] re-couples the independent constraints of detection and description of multi-constrained feature learning with a mutual weighting strategy and thus does not directly suppress the probability of detecting an ambiguous feature. SuperGlue [30] constructs a Graph Neural Network (GNN) for jointly finding correspondences and rejecting non-matchable points by treating the feature matching problem as solving a differentiable optimal transport problem.…”

Section: Related Workmentioning

confidence: 99%

“…The latest algorithms include LoFTR [31], which consists of two coarse matching strategies, the dual-softmax method adopted by the original text, called LoFTR-DS, and the optimal transport method adopted by SuperGlue [30], called LoFTR-OT, and we tested them against the outdoor data trained with each of the two algorithms. In addition, we compared our algorithm to the latest ReDFeat [29] multimodal algorithm, which uses the corresponding VIS-NIR weights, and the multimodal module uses the best result among the weights. The test results in this study were obtained using a computer with a 4-core, 8-thread Intel i3-12100F@3.3/4.3 GHz CPU and an 8GB RTX4060 GPU.…”

Section: Baseline and Metricsmentioning

confidence: 99%

Radiation-Variation Insensitive Coarse-to-Fine Image Registration for Infrared and Visible Remote Sensing Based on Zero-Shot Learning

Li,

Bi,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Infrared and visible remote sensing image registration is significant for utilizing remote sensing images to obtain scene information. However, it is difficult to establish a large number of correct matches due to the difficulty in obtaining similarity metrics due to the presence of radiation variation between heterogeneous sensors, which is caused by different imaging principles. In addition, the existence of sparse textures in infrared images as well as in some scenes and the small number of relevant trainable datasets also hinder the development of this field. Therefore, we combined data-driven and knowledge-driven methods to propose a Radiation-variation Insensitive, Zero-shot learning-based Registration (RIZER). First, RIZER, as a whole, adopts a detector-free coarse-to-fine registration framework, and the data-driven methods use a Transformer based on zero-shot learning. Next, the knowledge-driven methods are embodied in the coarse-level matches, where we adopt the strategy of seeking reliability by introducing the HNSW algorithm and employing a priori knowledge of local geometric soft constraints. Then, we simulate the matching strategy of the human eye to transform the matching problem into a model-fitting problem and employ a multi-constrained incremental matching approach. Finally, after fine-level coordinate fine tuning, we propose an outlier culling algorithm that only requires very few iterations. Meanwhile, we propose a multi-scene infrared and visible remote sensing image registration dataset. After testing, RIZER achieved a correct matching rate of 99.55% with an RMSE of 1.36 and had an advantage in the number of correct matches, as well as a good generalization ability for other multimodal images, achieving the best results when compared to some traditional and state-of-the-art multimodal registration algorithms.

show abstract

“…Hu Moments refer to a collection of mathematical descriptors or moments utilized for characterizing an object's shape in the fields of image analysis and pattern recognition. 16 The role of normalized central moments is to provide a standardized measure of an object's shape or distribution of intensity values in an image, independent of factors like translation, scale, and rotation. Variation caused by these factors can be eliminated by normalizing the central moments, making the moments more robust and suitable for comparison and recognition tasks.…”

Section: Flaws Characteristics Extraction Of Surface Defects and Data...mentioning

confidence: 99%

Research on surface defect identification of steel balls based on improved K-CV parameter optimization support vector machine

Li,

Ren,

Feng

2023

Advances in Mechanical Engineering

View full text Add to dashboard Cite

Surface defects generated during the production process of steel balls can lead to bearing failures, which makes it crucial to promptly detect and classify these defects. Defects classify is helpful for analysis and improving the production process. An algorithm that incorporates K-fold cross-validation (K-CV) with improved grid search is proposed to optimize the parameters of SVM, in order to detect surface defects with steel balls. Principal Component Analysis (PCA) was employed to reduce the dimensionality of the effective features data. The K-CV algorithm was employed in conjunction with an improved grid search method to find the optimal parameters “c” and “g.” This approach not only reduced the search time but also diminished the influence of individual samples on the model, thereby enhancing its robustness and ultimately improving the classification accuracy. The model’s performance was evaluated using a confusion matrix, and a comparison was made with three other machine learning models. The experimental results demonstrated the effectiveness of the proposed algorithm in classifying defects on highly reflective metal surfaces such as steel balls. The model achieved a classification accuracy of 97.15%.

show abstract

ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning

Cited by 21 publications

References 41 publications

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Radiation-Variation Insensitive Coarse-to-Fine Image Registration for Infrared and Visible Remote Sensing Based on Zero-Shot Learning

Research on surface defect identification of steel balls based on improved K-CV parameter optimization support vector machine

Contact Info

Product

Resources

About