Improved Compressed String Dictionaries

Abstract

We introduce a new family of compressed data structures to efficiently store and query large string dictionaries in main memory. Our main technique is a combination of hierarchical Front-coding with ideas from longest-common-prefix computation in suffix arrays. Our data structures yield relevant space-time tradeoffs in realworld dictionaries. We focus on two domains where string dictionaries are extensively used and efficient compression is required: URL collections, a key element in Web graphs and applications such as Web mining; and collections of URIs and literals, the basic components of RDF datasets. Our experiments show that our data structures achieve better compression than the state-of-theart alternatives while providing very competitive query times.

Space/time-efficient RDF stores based on circular suffix sorting

“…Our current implementation is designed to handle integer-based triples, so it requires an external dictionary to handle the mapping. As future work, we plan to integrate RDFCSA with some compressed dictionary [18], [19], [40] in order to provide efficient mappings between strings and ids. Another choice is to integrate it in the HDT library (http://rdfhdt.org), which already provides the needed string dictionaries.…”

“…Note that any of the other variants, including RDFCSA, could be complemented with a compact string dictionary that follows the encoding proposed for HDT. Solutions like HashDAC-RP [19] can answer string-to-id and id-to-string translations in a few microseconds per operation (typically 1-4 in URI and literal dictionaries such as those required in DBpedia [19], [40]), and would move solutions based on ids an extra 60% to the right in our plots. Since each triple-pattern requires at most 3 string-to-id translations per query, and at most 3 id-to-string operations per returned result (i.e., at most 2 translations if we omit the (?s, ?p, ?o) triple-pattern), query times would be increased by a few microseconds per result.…”