Joint Inverted Indexing

Abstract

Inverted indexing is a popular non-exhaustive solution to large scale search. An inverted file is built by a quantizer such as k-means or a tree structure. It has been found that multiple inverted files, obtained by multiple independent random quantizers, are able to achieve practically good recall and speed.Instead of computing the multiple quantizers independently, we present a method that creates them jointly. Our method jointly optimizes all codewords in all quantizers. Then it assigns these codewords to the quantizers. In experiments this method shows significant improvement over various existing methods that use multiple independent quantizers. On the one-billion set of SIFT vectors, our method is faster and more accurate than a recent state-of-the-art inverted indexing method.

PQTable: Fast Exact Asymmetric Distance Neighbor Search for Product Quantization Using Hash Tables

“…To handle large (e.g., N = 10 9 ) databases, short-code-based inverted indexing systems [1,23,7,15,3,4] have been proposed, which are currently state-of-the-art ANN methods (see Fig. 1b).…”

“…Such systems operate in two stages: (1) coarse quantization and (2) reranking via short codes. A database vector is first assigned to a bucket using a coarse quantizer (e.g., k-means [13], multiple k-means [23], or Cartesian products [1,12]), then compressed to a short code using PQ [13] or its extensions [17,7] and finally stored as a posting list 1 . In the retrieval phase, a query vector is assigned to the nearest buckets by the coarse quantizer, then associated items in corresponding posting lists are traversed, with the nearest one being reranked via ADC 2 .…”

Large-Scale Image Retrieval with Bag-of-Words and k-NN Re-Ranking

“…On the other hand, to address the problem of vocabulary correlation, the literature [13] present to create the vocabularies jointly and decrease correlation from the view of vocabulary generation.…”

Scalable Image Search with Multiple Index Tables

“…However, the threshold is not easily chosen among various image datasets. A joint indexing approach [9] was shown to optimize the quantization process. They indicate that multiple quantizers are effective in improving the indexing recall.…”

“…Instead of being trapped in such dilemmas, researchers have explored many different ways to improve search performance [5][13] [16][17] [9]. Jegou et al [5] proposed the Vector of Locally Aggregated Descriptors(VLAD) which are derived from local descriptors to replace the bag-of-words histogram.…”