A candidate filtering mechanism for fast top-k query processing on modern cpus

Abstract

A large amount of research has focused on faster methods for finding top-k results in large document collections, one of the main scalability challenges for web search engines. In this paper, we propose a method for accelerating such top-k queries that builds on and generalizes methods recently proposed by several groups of researchers based on Block-Max Indexes [15,10,13]. In particular, we describe a system that uses a new filtering mechanism, based on a combination of block maxima and bitmaps, that radically reduces the number of documents that have to be further evaluated. Our filtering mechanism exploits the SIMD processing capabilities of current microprocessors, and it is optimized through caching policies that select and store suitable filter structures based on properties of the query load. Our experimental evaluation shows that the mechanism results in very significant speed-ups for disjunctive top-k queries under several state-of-the-art algorithms, including a speed-up of more than a factor of 2 over the fastest previously known methods.

Document Prioritization for Scalable Query Processing

“…In terms of the efficiency, the proposed method can reduce the query processing time by a factor of 2 for longer queries or larger K on Web collections, compared with the state of the art query processing technique, i.e., block-max index methods [16,18]. It can also consistently improve the efficiency even for smaller K on Twitter collections, but the speed up is not as big as on the Web collections.…”

“…A document needs to be fully evaluated based on the contribution of all query terms that occur in the document before moving to the next one [6,16,18,22,30,33].…”

“…An alternative solution is to improve the disjunctive mode through dynamic pruning techniques [6,12,[16][17][18]27,29,33]. Instead of exhaustively evaluating every document that matches at least one query term, dynamic pruning techniques evaluate only a smaller set of documents that have greater potential to make to top-K results and bypass the evaluation for the rest of the documents.…”

“…With the increasing amount of online information and the rapid growing number of queries, commercial search engines have leveraged many performance optimization techniques, including parallel computing [5], caching [4], index compression [1,10,36,38] and dynamic pruning [6,12,16,18,27,29,33], to meet strict performance constraints -high throughput and low latency. We now provide some background about a few most related topics to our study.…”

Fast top-k preserving query processing using two-tier indexes

“…Special optimizations to the DAAT methods, including BMW, were proposed by Dimopoulos, Nepomnyachiy, and Suel (2013) . Authors propose the use of a set of block-max and posting-bitmap structures that can be used to transparently accelerate any DAAT index traversal algorithm.…”