Performance of compressed inverted list caching in search engines

Zhang, Jiangong; Long, Xiaohui; Suel, Torsten

doi:10.1145/1367497.1367550

Cited by 190 publications

(135 citation statements)

References 19 publications

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…Essentially, it is possible to cache query results [Fagni et al 2006;Gan and Suel 2009;Markatos 2001], a portion of the inverted index [Baeza-Yates et al 2008;Zhang et al 2008], or a combination of both [BaezaYates and Jonassen 2012;Baeza-Yates and Saint-Jean 2003;Baeza-Yates et al 2008;Long and Suel 2005;Saraiva et al 2001]. Furthermore, caching strategies for documents and snippets are also proposed [Ceccarelli et al 2011;Tsegay et al 2009;Turpin et al 2007].…”

Section: Search Results Cachingmentioning

confidence: 99%

Second Chance: A Hybrid Approach for Dynamic Result Caching in Search Engines

Altıngövde

Ozcan

Cambazoğlu

et al. 2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Web search engines are known to cache the results of previously issued queries. The stored results typically contain the document summaries and some data that is used to construct the final search result page returned to the user. An alternative strategy is to store in the cache only the result document IDs, which take much less space, allowing results of more queries to be cached. These two strategies lead to an interesting trade-off between the hit rate and the average query response latency. In this work, in order to exploit this trade-off, we propose a hybrid result caching strategy where a dynamic result cache is split into two sections: an HTML cache and a docID cache. Moreover, using a realistic cost model, we evaluate the performance of different result prefetching strategies for the proposed hybrid cache and the baseline HTML-only cache. Finally, we propose a machine learning approach to predict singleton queries, which occur only once in the query stream. We show that when the proposed hybrid result caching strategy is coupled with the singleton query predictor, the hit rate is further improved.

show abstract

Section: Search Results Cachingmentioning

confidence: 99%

Second Chance: A Hybrid Approach for Dynamic Result Caching in Search Engines

Altıngövde

Ozcan

Cambazoğlu

et al. 2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…As a variable byte codec, we use the Apache Hadoop 3 implementation, VInt. For classical PFOR [9,25], we use the implementation in the Kamikaze project 4 . Finally, the JavaFastPFOR package 5 provides implementations of several codecs, namely FOR, NewPFD, OptPFD, FastPFOR and Simple 16 [26] (a member of the Simple family).…”

Section: Compression Codecsmentioning

confidence: 99%

On Inverted Index Compression for Search Engine Efficiency

Catena

Macdonald

Ounis

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Efficient access to the inverted index data structure is a key aspect for a search engine to achieve fast response times to users' queries. While the performance of an information retrieval (IR) system can be enhanced through the compression of its posting lists, there is little recent work in the literature that thoroughly compares and analyses the performance of modern integer compression schemes across different types of posting information (document ids, frequencies, positions). In this paper, we experiment with different modern integer compression algorithms, integrating these into a modern IR system. Through comprehensive experiments conducted on two large, widely used document corpora and large query sets, our results show the benefit of compression for different types of posting information to the space-and time-efficiency of the search engine. Overall, we find that the simple Frame of Reference compression scheme results in the best query response times for all types of posting information. Moreover, we observe that the frequency and position posting information in Web corpora that have large volumes of anchor text are more challenging to compress, yet compression is beneficial in reducing average query response times.

show abstract

“…Previous works [65,68,66,69,70,67] have focussed solely on two factors, the compression ratio and the decompression performance, disregarding the compression performance. While decompression performance is essential for query throughput, compression performance is crucial for update throughput.…”

Section: High-performance Compression For Node Indexing Schemementioning

confidence: 99%