Self-Supervised Foundation Model for Template Matching

Abstract

Finding a template location in query image is a fundamental problem in many computer vision applications, such as localization of known objects, image registration, image matching and object tracking. Currently available methods fail when not enough training data is available or big variations in the textures, different modalities and weak visual features exist in the images, leading to limited applications on real-world tasks. We introduce Self-Supervised Foundation Model for Template Matching (Self-TM), a novel end-to-end approach for self-supervised learning template matching. The idea behind Self-TM is to learn hierarchical features incorporating localization properties from images without any annotations. As going deeper in the convolutional neural network (CNN) layers, their filters begin to react to more complex structures and their receptive fields increase. This leads to loss of localization information in contrast to the early layers. The hierarchical propagation of the last layers activations back to the first layer results in precise template localization. Due to its zero-shot generalization capabilities on tasks such as image retrieval, dense template matching and sparse image matching our pre-trained model can be classified as a foundation one.