Object matching between visible and infrared images using a Siamese network

Abstract

In this study, we propose a method for object matching between visible and infrared images. We consider object matching between visible and infrared images as a computational patch-matching problem, and the main solution to this problem involves calculating the similarities of the relationships between the objects in the target and search images. Therefore, in this study, we propose a Siamese neural network, which takes a pair of visible and infrared images as the input. Our proposed Siamese network comprises a convolutional neural network (CNN) to ensure the effective extraction of features from visible and infrared images. The CNN comprises convolutional and pooling layers without padding. By calculating the cross-correlation of the objects in the visible image and those in the entire infrared image, we regard the parts with the highest similarity as the matched targets. During the training process, we use focal loss to solve the problem of the imbalance between the positive and negative samples in the dataset, after which we use interpolation to determine the locations of the target patches in the infrared images. We then conduct experiments on different classes of targets, and the results demonstrate that our proposed approach achieves greater accuracy and precision than other methods.