Exploiting SIFT Descriptor for Rotation Invariant Convolutional Neural Network

Abstract

This paper presents a novel approach to exploit the distinctive invariant features in convolutional neural network. The proposed CNN model uses Scale Invariant Feature Transform (SIFT) descriptor instead of the maxpooling layer. Max-pooling layer discards the pose, i.e., translational and rotational relationship between the low-level features, and hence unable to capture the spatial hierarchies between low and high level features. The SIFT descriptor layer captures the orientation and the spatial relationship of the features extracted by convolutional layer. The proposed SIFT Descriptor CNN therefore combines the feature extraction capabilities of CNN model and rotation invariance of SIFT descriptor. Experimental results on the MNIST and fashionMNIST datasets indicates reasonable improvements over conventional methods available in literature.

SIFT-CNN: When Convolutional Neural Networks Meet Dense SIFT Descriptors for Image and Sequence Classification

“…In this manner, many different approaches are proposed for the calculation of the local descriptors, either exploiting key-point SIFT [ 20 , 21 ] or jointly exploited with dense SIFT features [ 22 ]. Besides, the fusion method is varied from a simple concatenation to more sophisticated attention mechanisms [ 18 , 23 , 24 ]. Additionally, the previous dual-stream logic is modified by redoubling each stream and implementing a Siamese scheme [ 25 ].…”