An Experimental Study of Index Compression and DAAT Query Processing Methods

Mallia, Antonio; Siedlaczek, Michał; Suel, Torsten

doi:10.1007/978-3-030-15712-8_23

Cited by 26 publications

(28 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the underlying query type will generate a lot of skipping, the time/space trade-off changes considerably. A recent study from Mallia et al [175] had similar conclusions, though a different set of compression schemes was explored. In any case, it is important to highlight that the best compression codec ultimately depends on the requirements of the given search system.…”

Section: Bic [181]mentioning

confidence: 83%

Managing tail latency in large scale information retrieval systems

Mackenzie

2020

SIGIR Forum

View full text Add to dashboard Cite

As both the availability of internet access and the prominence of smart devices continue to increase, data is being generated at a rate faster than ever before. This massive increase in data production comes with many challenges, including efficiency concerns for the storage and retrieval of such large-scale data. However, users have grown to expect the sub-second response times that are common in most modern search engines, creating a problem --- how can such large amounts of data continue to be served efficiently enough to satisfy end users? This dissertation investigates several issues regarding tail latency in large-scale information retrieval systems. Tail latency corresponds to the high percentile latency that is observed from a system --- in the case of search, this latency typically corresponds to how long it takes for a query to be processed. In particular, keeping tail latency as low as possible translates to a good experience for all users, as tail latency is directly related to the worst-case latency and hence, the worst possible user experience. The key idea in targeting tail latency is to move from questions such as "what is the median latency of our search engine?" to questions which more accurately capture user experience such as "how many queries take more than 200 ms to return answers?" or "what is the worst case latency that a user may be subject to, and how often might it occur?" While various strategies exist for efficiently processing queries over large textual corpora, prior research has focused almost entirely on improvements to the average processing time or cost of search systems. As a first contribution, we examine some state-of-the-art retrieval algorithms for two popular index organizations, and discuss the trade-offs between them, paying special attention to the notion of tail latency. This research uncovers a number of observations that are subsequently leveraged for improved search efficiency and effectiveness. We then propose and solve a new problem, which involves processing a number of related query variations together, known as multi-queries , to yield higher quality search results. We experiment with a number of algorithmic approaches to efficiently process these multi-queries, and report on the cost, efficiency, and effectiveness trade-offs present with each. Finally, we examine how predictive models can be used to improve the tail latency and end-to-end cost of a commonly used multi-stage retrieval architecture without impacting result effectiveness. By combining ideas from numerous areas of information retrieval, we propose a prediction framework which can be used for training and evaluating several efficiency/effectiveness trade-off parameters, resulting in improved trade-offs between cost, result quality, and tail latency.

show abstract

Section: Bic [181]mentioning

confidence: 83%

Managing tail latency in large scale information retrieval systems

Mackenzie

2020

SIGIR Forum

View full text Add to dashboard Cite

show abstract

“…Shan et al [15] show that MaxScore variants (e.g. BMM, LBMM) outperform other dynamic pruning strategies for long queries, and recently Mallia et al [2] report a similar finding over a range of popular index encodings.…”

Section: Related Workmentioning

confidence: 90%

“…In this paper, we apply dynamic pruning to structural search. As structure search has more query terms in general, we focus on a MaxScore-like strategy suggested by [2,15], since they do not need to sort query terms at merge iterations (which is expensive for long queries). Our approach is different from the original MaxScore, as upperbound scores are also calculated from the query tree representation.…”

Section: Methodsmentioning

confidence: 99%

“…Given query hit node q1 in Fig. 3, function qryNodeMatch returns max n∈T d ν(G (1,n) ) = max(min(5, 2) + min (1,2), min(5, 3)) = 3…”

Section: Methodsmentioning

confidence: 99%

“…Dynamic pruning has been recognized as an effective way to reduce query processing times [2,8,13,18]. Dynamic pruning speeds up query processing by skipping scoring calculations or avoiding unnecessary reads for documents which are unlikely to be ranked in the top K results.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Accelerating Substructure Similarity Search for Formula Retrieval

Zhong

Rohatgi

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Formula retrieval systems using substructure matching are effective, but suffer from slow retrieval times caused by the complexity of structure matching. We present a specialized inverted index and rank-safe dynamic pruning algorithm for faster substructure retrieval. Formulas are indexed from their Operator Tree (OPT) representations. Our model is evaluated using the NTCIR-12 Wikipedia Formula Browsing Task and a new formula corpus produced from Math StackExchange posts. Our approach preserves the effectiveness of structure matching while allowing queries to be executed in real-time.

show abstract

Efficient query processing techniques for next-page retrieval

2022

View full text Add to dashboard Cite

In top-k ranked retrieval the goal is to efficiently compute an ordered list of the highest scoring k documents according to some stipulated similarity function such as the well-known BM25 approach. In most implementation techniques a min-heap of size k is used to track the top scoring candidates. In this work we consider the question of how best to retrieve the second page of search results, given that a first page has already been computed; that is, identification of the documents at ranks $$k+1$$ k + 1 to 2k for some query. Our goal is to understand what information is available as a by-product of the first-page scoring, and how it can be employed to accelerate the second-page computation, assuming that the second-page of results is required for only a fraction of the query load. We propose a range of simple, yet efficient, next-page retrieval techniques which are suitable for accelerating Document-at-a-Time mechanisms, and demonstrate their performance on three large text collections.

show abstract

An Experimental Study of Index Compression and DAAT Query Processing Methods

Cited by 26 publications

References 48 publications

Managing tail latency in large scale information retrieval systems

Managing tail latency in large scale information retrieval systems

Accelerating Substructure Similarity Search for Formula Retrieval

Efficient query processing techniques for next-page retrieval

Contact Info

Product

Resources

About