Optimizing top-k document retrieval strategies for block-max indexes

Dimopoulos, C; Nepomnyachiy, Sergey; Suel, Torsten

doi:10.1145/2433396.2433412

Cited by 59 publications

(38 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We focus here on the DAAT versions of the algorithm in [24,17,13]. Maxscore splits the query terms into essential and nonessential terms as follows.…”

Section: Et Algorithmsmentioning

confidence: 99%

“…This structure stores the highest impact score for each block in the inverted lists, where the blocks are defined by the index compression method. Recently in [13], the blocks of the Block-Max structures were decoupled from the index compression and defined on docID space rather than posting distribution. We will refer to Block-Max structures whose blocks are defined by postings as posting-oriented, while the ones defined on docID space as docID-oriented.…”

Section: Block-max Indexesmentioning

confidence: 99%

“…Content of this Paper: Recent work by several groups of researchers [10,15,19,13] has shown significant improvements in performance based on a Block-Max Index, first proposed in [10,15]. This is an index structure that stores maximum scores for groups of postings, thus allowing quick skipping, without full evaluation, of sets of postings with low scores.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we propose a method for accelerating such top-k queries that builds on and generalizes methods recently proposed by several groups of researchers based on Block-Max Indexes [15,10,13]. In particular, we describe a system that uses a new filtering mechanism, based on a combination of block maxima and bitmaps, that radically reduces the number of documents that have to be further evaluated.…”

mentioning

confidence: 99%

See 3 more Smart Citations

A candidate filtering mechanism for fast top-k query processing on modern cpus

Dimopoulos

Nepomnyachiy

Suel

2013

Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

A large amount of research has focused on faster methods for finding top-k results in large document collections, one of the main scalability challenges for web search engines. In this paper, we propose a method for accelerating such top-k queries that builds on and generalizes methods recently proposed by several groups of researchers based on Block-Max Indexes [15,10,13]. In particular, we describe a system that uses a new filtering mechanism, based on a combination of block maxima and bitmaps, that radically reduces the number of documents that have to be further evaluated. Our filtering mechanism exploits the SIMD processing capabilities of current microprocessors, and it is optimized through caching policies that select and store suitable filter structures based on properties of the query load. Our experimental evaluation shows that the mechanism results in very significant speed-ups for disjunctive top-k queries under several state-of-the-art algorithms, including a speed-up of more than a factor of 2 over the fastest previously known methods.

show abstract

“…We focus here on the DAAT versions of the algorithm in [24,17,13]. Maxscore splits the query terms into essential and nonessential terms as follows.…”

Section: Et Algorithmsmentioning

confidence: 99%

Section: Block-max Indexesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

A candidate filtering mechanism for fast top-k query processing on modern cpus

Dimopoulos

Nepomnyachiy

Suel

2013

Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

show abstract

“…e authors reported an average query response time reduction of BMW compared to Wand of 64% -67%. Experiments in [5] reported a reduction of 66% by BMW with respect to Wand. A more recent work [2] explored the performance of BMW compared to Wand on di erent document collection and di erent query logs.…”

Section: Query Processingmentioning

confidence: 98%

Upper Bound Approximation for BlockMaxWand

Macdonald

Tonellotto

2017

Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval

View full text Add to dashboard Cite

BlockMaxWand is a recent advance on the Wand dynamic pruning technique, which allows e cient retrieval without any e ectiveness degradation to rank K. However, while BMW uses docid-sorted indices, it relies on recording the upper bound of the term weighting model scores for each block of postings in the inverted index. Such a requirement can be disadvantageous in situations such as when an index must be updated. In this work, we examine the appropriateness of upper-bound approximation -which have previously been shown suitable for Wand-in providing e cient retrieval for BMW. Experiments on the ClueWeb12 category B13 corpus using 5000 queries from a real search engine's query log demonstrate that BMW still provides bene ts w.r.t. Wand when approximate upper bounds are used, and that, if approximations on upper bounds are tight, BMW with approximate upper bounds can provide e ciency gains w.r.t. Wand with exact upper bounds, in particular for queries of short to medium length.

show abstract

Compact inverted index storage using general‐purpose compression libraries

Petri

Moffat

2018

Softw Pract Exp

View full text Add to dashboard Cite

Efficient storage of large inverted indexes is one of the key technologies that support current web search services. Here we re-examine mechanisms for representing document-level inverted indexes and within-document term frequencies, including comparing specialized methods developed for this task against recent fast implementations of general-purpose adaptive compression techniques. Experiments with the Gov2-URL collection and a large collection of crawled news stories show that standard compression libraries can provide compression effectiveness as good as or better than previous methods, with decoding rates only moderately slower than reference implementations of those tailored approaches. This surprising outcome means that high-performance index compression can be achieved without requiring the use of specialized implementations. KEYWORDSindex compression, inverted index, web search 974

show abstract

Optimizing top-k document retrieval strategies for block-max indexes

Cited by 59 publications

References 25 publications

A candidate filtering mechanism for fast top-k query processing on modern cpus

A candidate filtering mechanism for fast top-k query processing on modern cpus

Upper Bound Approximation for BlockMaxWand

Compact inverted index storage using general‐purpose compression libraries

Contact Info

Product

Resources

About