Practical compressed string dictionaries

Martínez‐Prieto, Miguel A.; Brisaboa, Nieves R.; Cánovas, Rodrigo; Claude, Francisco; Navarro, Gonzalo

doi:10.1016/j.is.2015.08.008

Cited by 45 publications

(46 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…12,[45][46][47][48] With respect to DA dictionaries, BASE and CHECK use many bits since they are pointer-based arrays. Therefore, we will investigate efficient compression methods for BASE and CHECK in dynamic DAs.…”

Section: Discussionmentioning

confidence: 99%

Practical rearrangement methods for dynamic double‐array dictionaries

et al. 2017

View full text Add to dashboard Cite

SummaryDouble-array structures have been widely used to implement dictionaries with string keys. Although the space efficiency of dynamic double-array dictionaries tends to decrease with key updates, we can still maintain high efficiency using existing methods. However, these methods have practical problems of time and functionality. This paper presents several efficient rearrangement methods to solve these problems.Through experiments using real-world datasets, we demonstrate that the proposed rearrangement methods are much more practical than existing methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Practical rearrangement methods for dynamic double‐array dictionaries

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Both datasets have been obtained from the Webgraph framework [3]. The UK dataset has been used in previous work as a baseline for URL compression [1,7,14]. The Arabic dataset is included for better confirmation of the performance of each solution in different Web graphs.…”

Section: Methodsmentioning

confidence: 99%

“…• PFC, RPFC and RPHTFC are some of the differential encoding techniques based on Front-coding [14] described in Section 2. PFC is the plain solution, RPFC uses Re-Pair to compress buckets.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Improved Compressed String Dictionaries

Brisaboa

Cerdeira-Pena

Bernardo

et al. 2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

We introduce a new family of compressed data structures to efficiently store and query large string dictionaries in main memory. Our main technique is a combination of hierarchical Front-coding with ideas from longest-common-prefix computation in suffix arrays. Our data structures yield relevant space-time tradeoffs in realworld dictionaries. We focus on two domains where string dictionaries are extensively used and efficient compression is required: URL collections, a key element in Web graphs and applications such as Web mining; and collections of URIs and literals, the basic components of RDF datasets. Our experiments show that our data structures achieve better compression than the state-of-theart alternatives while providing very competitive query times.

show abstract

“…This significantly reduces the number of comparisons to be made on binary searches, at the price of a small space overhead. These indexes are implemented using compressed string dictionaries [44].…”

Section: A2 Slp-based Self Indexesmentioning

confidence: 99%

Universal indexes for highly repetitive document collections

Claude

Fariña²,

Martínez‐Prieto³

et al. 2016

Information Systems

Self Cite

View full text Add to dashboard Cite

Indexing highly repetitive collections has become a relevant problem with the emergence of large repositories of versioned documents, among other applications. These Collections may reach huge sizes, but are formed mostly of documents that are near-copies of others. Traditional techniques for indexing these collections fail to properly exploit their regularities in order to reduce space. We introduce new techniques for compressing inverted indexes that exploit this near copy regularity. They are based on run-length, Lempel-Ziv, or grammar compression of the differential inverted lists, instead of the usual practice of gap-encoding them. We show that, in this highly repetitive setting, our compression methods significantly reduce the space obtained with classical techniques, at the price of moderate slowdowns. Moreover, our best methods are universal, that is, they do not need to know the versioning structure of the collection, nor that a clear versioning structure even exists. We also introduce compressed self-indexes in the comparison. These are designed for general strings (not only natural language texts) and represent the text collection plus the index structure (not an inverted index) in integrated form. We show that these techniques can compress much further, using a small fraction of the space required by our new inverted indexes. Yet they are orders of magnitude slower. (C) 2016 Elsevier Ltd. All rights reservedEuropean Union 690941 Fondecyt (Conicyt, Chile) 1-140796 MINECO (PGE) TIN2013-47090-C3-3-P TIN2015-69951-R TIN2013-46238-C4-3-R MINECO (FEDER) TIN2013-47090-C3-3-P TIN2015-69951-R TIN2013-46238-C4-3-R ICT COST Action IC1302 Xunta de Galicia (FEDER) (Spain) GRC2013/05

show abstract

Practical compressed string dictionaries

Cited by 45 publications

References 74 publications

Practical rearrangement methods for dynamic double‐array dictionaries

Practical rearrangement methods for dynamic double‐array dictionaries

Improved Compressed String Dictionaries

Universal indexes for highly repetitive document collections

Contact Info

Product

Resources

About