Coupled Binary Embedding for Large-Scale Image Retrieval

Abstract

Abstract-Visual matching is a crucial step in image retrieval based on the bag-of-words (BoW) model. In the baseline method, two keypoints are considered as a matching pair if their SIFT descriptors are quantized to the same visual word. However, the SIFT visual word has two limitations. First, it loses most of its discriminative power during quantization. Second, SIFT only describes the local texture feature. Both drawbacks impair the discriminative power of the BoW model and lead to false positive matches. To tackle this problem, this paper proposes to embed multiple binary features at indexing level. To model correlation between features, a multi-IDF scheme is introduced, through which different binary features are coupled into the inverted file. We show that matching verification methods based on binary features, such as Hamming embedding, can be effectively incorporated in our framework. As an extension, we explore the fusion of binary color feature into image retrieval. The joint integration of the SIFT visual word and binary features greatly enhances the precision of visual matching, reducing the impact of false positive matches. Our method is evaluated through extensive experiments on four benchmark datasets (Ukbench, Holidays, DupImage, and MIR Flickr 1M). We show that our method significantly improves the baseline approach. In addition, largescale experiments indicate that the proposed method requires acceptable memory usage and query time compared with other approaches. Further, when global color feature is integrated, our method yields competitive performance with the state-of-the-arts.Index Terms-Feature fusion, coupled binary embedding, multi-IDF, image retrieval.

Large Scale Multimedia Management: Recent Challenges

“…Binary coding aims to present feature vectors by a compact binary code. For example, in [3], the authors present dual binary embedding applied lo large scale images retrieval. The system uses multiples binary features extracted from SIFT (Scale Invariant Feature Transform) feature and a multi-IDF (Inverse Document Frequency) scheme allowing the association of binary features to the inverted file.…”

Fusion of Global and Local Features with Multi-Inverted Indices for Efficient Image Retrieval

“…In practice, additional data, such as other feature descriptors or cues, can be embedded in the inverted index (together with image IDs) for refined re-ranking, such as Hamming embedding [61], binary embedding [21], semantic-aware coindexing [57], and IR embedding [29]. The embedded information is typically compact (e.g.…”

“…The inverted index can be extended to multidimensional cases, aka inverted multi-index (IMI) [20]. If each index dimension corresponds to a different feature, then IMIlike structures can be used for feature fusion, aka coupled indexing [21]. For a two-dimensional IMI with dimensions M 1 and M 2 , the time complexity is reduced to O(N/M 1 /M 2 ), but the space complexity is increased to O(M 1 M 2 ).…”

Image Retrieval Based on ConvNets and Hashing Algorithm

“…In practice, SIFT and SURF are trade-off options because SIFT yields better accuracy results but slower inference. In 8,9 , they implemented a new approach when creating bag-of-words (BOW) for a large database; despite choosing only SIFT, they decided to add binary descriptions and achieved 79.6% accuracy on a holiday dataset. Deep Learning visual descriptor: Along with lowlevel features, the features extracted by these approaches above seem to exploit only a small amount of image information.…”