Joint Detection and Matching of Feature Points in Multimodal Images

Baruch, Elad Ben; Keller, Yosi

doi:10.1109/tpami.2021.3092289

Cited by 10 publications

(10 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose a pseudo-Siamese network structure, which contains two independent but identical convolutional streams and processes two different input data in parallel. The pseudo-Siamese network has been proved to be effective in image matching [40] and change detection [41].…”

Section: A Network Architecturementioning

confidence: 99%

A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Yang

Peng

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Due to technology and cost limitations, it is challenging to obtain high temporal and spatial resolution images from a single satellite spectrometer, which significantly limits the specific application of such remote sensing images in earth science. To solve the problem that the existing algorithms cannot effectively balance the spatial detail preservation and spectral change reconstruction, a pseudo-Siamese deep convolutional neural network (PDCNN) for spatiotemporal fusion is proposed in this article. The method proposes a pseudo-Siamese network framework model for fusion. This framework has two independent and equal feature extraction streams, but the weights are not shared. The two feature extraction streams process the image information at the previous and later moments and reconstruct the fine image of the corresponding time to fully extract the image information at different times. In the feature extraction stream, the multiscale mechanism and dilated convolution of flexible perception are designed, which can flexibly obtain feature image information and improve the model reconstruction accuracy. In addition, an attention mechanism is introduced to improve the weight of the crucial information for the remote sensing images. Adding a residual connection enhances the reuse of the initial feature information in shallow networks and reduces the loss of feature information in deep networks. Finally, the fine images obtained from the two feature extraction streams are weighted and fused to obtain the final predicted image. The subjective and objective results demonstrate that the PDCNN can effectively reconstruct the fusion image with higher quality.

show abstract

Section: A Network Architecturementioning

confidence: 99%

A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Yang

Peng

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Similar to the 2channel network that merges two image patches in pixel level, Quan et al [40] proposed the SCFDM that splices two image patches along the spatial dimension. Hybrid [18] combines Siamese and pre-Siamese build four-branch network for multi-modal image matching. Moreshet et al [35] used multiscale siamese network extract feature map, combined with transformer to obtain image global information and improve network performance.…”

Section: Related Workmentioning

confidence: 99%

“…In contrast to the traditional algorithm, the learning-based algorithms have demonstrated a powerful capacity for high-level feature extraction and similarity measurement, achieving impressive performance even on cross-spectral patch matching. They can generally fall into two groups: descriptor learning [16,28,29,30,31,32] and metric learning [17,33,34,18,35]. Despite the success of existing methods like the hybrid approach [18] in achieving good results on specific datasets (Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Underwater Target Noise Recognition and Classification Technology based on Multi-Classes Feature Fusion

Zhang

Wang²,

Sun³

2020

JNWPU

View full text Add to dashboard Cite

As one of the main signal sources of underwater acoustic target recognition, the target noise signal is difficult to characterize the characteristics of the target from clearly comparing with the multi-sensor detection technology, which may lead to lower recognition rate and higher false alarm rate and seriously restricts the function of underwater acoustic detection system. In order to solve this problem, a multi-layers LSTM underwater acoustic target noise feature extraction model is established by using the long short term memory network. The information features such as time-domain envelope of target noise, Demon line spectrum and Mel frequency cepstrum coefficient are extracted, and a subset of multi-classes features is constructed. On this basis, the feature level fusion recognition and classification model based on the multi-classes features subset and the decision level fusion recognition and classification model based on the D-S evidence theory are established, and the above-mentioned models are tested by using the sample database. The difference of classification result between the multi-classes feature fusion and the single class feature recognition classification is compared, and the above model is tested and verified by using the relevant test data of port basin verification experiment. The correlation results show that the present intelligent recognition and classification method of underwater target noise based on the multi-classes feature fusion is more robust, and the recognition rate and false alarm rate of underwater target are better than those of single category feature discrimination method.

show abstract

“…

Point matching plays an important role in computer vision, such as object matching [1,2], image registration [3,4], autonomous navigation [5], 3D modeling [6] and so on. It aims to find the correspondences between two point sets and/or recover the transformation that map one point set to the other.

…”

mentioning

confidence: 99%

Point matching based on affine invariant centroid trees

Wang

Yan

Zhao³

et al. 2022

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Object detection can be formulated as a point matching problem when objects are modeled by point sets. Moments, which have been widely used for point matching, are limited to affine transformations as their support point sets cannot keep invariant. To address this problem, we developed an affine invariant centroid tree (AICT) to obtain a rigorous affine invariant support point set (SPS). The algorithm is constructed by a recursive process: the point set is first divided by the vector from the certain point to the centroid of the point set, and the centroids of subsets are used to generate vectors for renewed partitions. In addition, the centroids of the subsets are stored to form an AICT. The AICT represents the inherent structure of the point set. It is highly tolerant to noise and outliers due to the partitions on the whole point set. More importantly, it is affine invariant owing to the affine invariance of partition. Therefore, we can get rigorous affine invariant descriptors while moments are combined with AICT. The experimental results on synthesized and real data verify that our proposed algorithm outperforms the state-of-the-art point matching methods including shape context, iterative closet point, and the method adopting thin plate spline for rigid robust point matching (TPS-RPM).

show abstract

Joint Detection and Matching of Feature Points in Multimodal Images

Cited by 10 publications

References 45 publications

A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

A Pseudo-Siamese Deep Convolutional Neural Network for Spatiotemporal Satellite Image Fusion

Underwater Target Noise Recognition and Classification Technology based on Multi-Classes Feature Fusion

Point matching based on affine invariant centroid trees

Contact Info

Product

Resources

About