Rank/select operations on large alphabets

Combinatorial Pattern Matching

2006

We give new solutions to the Searchable Partial Sums with Indels problem. Given a sequence of n k-bit numbers, we present a structure taking kn + o(kn) bits of space, able of performing operations sum, search, insert, and delete, all in O(log n) worst-case time, for any k = O(log n). This extends previous results by Hon et al. [ISAAC 2003] achieving the same space and O(log n/ log log n) time complexities for the queries, yet offering complexities for insert and delete that are amortized and worse than ours, and supported only for k = O(1). Our result matches an existing lower bound for large values of k.We also give new solutions to the Dynamic Sequence problem. Given a sequence of n symbols in the range [1, σ] with binary zero-order entropy H 0 , we present a dynamic data structure that requires nH 0 + o(n log σ) bits of space, which is able of performing rank and select, as well as inserting and deleting symbols at arbitrary positions, in O(log n log σ) time. Our result is the first entropy-bound dynamic data structure for rank and select over general sequences.In the case σ = 2, where both previous problems coincide, we improve the dynamic solution of Hon et al. in that we compress the sequence. The only previous result with entropy-bound space for dynamic binary sequences is by Blandford and Blelloch [SODA 2004], which has the same complexities as our structure, but does not achieve constant 1 multiplying the entropy term in the space complexity.Finally, we present a new dynamic compressed full-text self-index, for a collection of texts over an alphabet of size σ, of overall length n and h-th order empirical entropy H h . The index requires nH h + o(n log σ) bits of space, for any h ≤ α log σ n and constant 0 < α < 1. It can count the number of occurrences of a pattern of length m in time O(m log n log σ). Each such occurrence can be reported in O(log 2 n log log n) time, and displaying a context of length ℓ from a text takes time O(log n(ℓ log σ + log n log log n)). Insertion/deletion of a text to/from the collection takes O(log n log σ) time per symbol. This significantly improves the space of a previous result by Chan et al. [CPM 2004] in exchange for a slight time complexity penalty. We achieve at the same time the first dynamic index requiring essentially nH h bits of space, and the first construction of a compressed full-text self-index within that working space. Previous results achieve at best O(nH h ) space with constants larger than 1 (Ferragina and Manzini [FOCS 2000], Arroyuelo and Navarro [ISAAC 2005]) and higher time complexities.An important result we prove in this paper is that the wavelet tree of the Burrows-Wheeler transform of a text, if compressed with a technique that achieves zero-order compression locally (e.g., Raman et al. [SODA 2002]), automatically achieves h-th order entropy space for any h. This unforeseen relation is essential for the results of the previous paragraph, but it also derives into significant simplifications on many existing static compressed full-text self-indexes...

mentioning

confidence: 99%

Dynamic Entropy-Compressed Sequences and Full-Text Indexes

Mäkinen

Combinatorial Pattern Matching

2006

“…Golynski, Munro, and Rao (GMR) [18] presented another representation that achieves time O(log log σ) for rank and access, and O(1) for select . Alternatively, they can achieve O(1) time for access, O(log log σ) for select , and O(log log σ log log log σ) for rank .…”

Section: Compact Data Structures For Sequencesmentioning

confidence: 99%

Extended Compact Web Graph Representations

Claude

Algorithms and Applications

2010

Abstract. Many relevant Web mining tasks translate into classical algorithms on the Web graph. Compact Web graph representations allow running these tasks on larger graphs within main memory. These representations at least provide fast navigation (to the neighbors of a node), yet more sophisticated operations are desirable for several Web analyses. We present a compact Web graph representation that, in addition, supports reverse navigation (to the nodes pointing to the given one). The standard approach to achieve this is to represent the graph and its transpose, which basically doubles the space requirement. Our structure, instead, represents the adjacency list using a compact sequence representation that allows finding the positions where a given node v is mentioned, and answers reverse navigation using that primitive. This is combined with a previous proposal based on grammar compression of the adjacency list. The combination yields interesting algorithmic problems. As a result, we achieve the smallest graph representation reported in the literature that supports direct and reverse navigation, and also obtain other variants that occupy relevant niches in the space/time tradeoff.

“…Wavelet trees [13] achieve n log σ+o(n) log σ bits of space while answering all the queries in O(log σ) time. Another interesting proposal [12], focused on large alphabets, achieves n log σ + no(log σ) bits of space and answers rank and access in O(log log σ) time, while select takes O(1) time. Another tradeoff within the same space [12] is O(1) time for access, O(log log σ) time for select, and O(log log σ log log log σ) time for rank.…”

Section: Succinct Data Structuresmentioning

confidence: 99%

“…We represent S B using wavelet trees [13], L with the structure for large alphabets [12], and X A and X B in compressed form [26]. Calling r = |R|, S B requires r log n 2 + o(r) log n 2 bits, L requires r log + r o(log ) bits (i.e., zero if = 1), and X A and X B use O(n 1 log r+n1 n1 + n 2 log r+n2 n2 ) + o(r + n 1 + n 2 ) = O(r) + o(n 1 + n 2 ) bits.…”

Section: Labeled Binary Relations With Range Queriesmentioning

confidence: 99%

See 1 more Smart Citation

Self-indexed Text Compression Using Straight-Line Programs

Claude

Mathematical Foundations of Computer Science 2009

2009

Abstract. Straight-line programs (SLPs) offer powerful text compression by representing a text T [1, u] in terms of a restricted context-free grammar of n rules, so that T can be recovered in O(u) time. However, the problem of operating the grammar in compressed form has not been studied much. We present a grammar representation whose size is of the same order of that of a plain SLP representation, and can answer other queries apart from expanding nonterminals. This can be of independent interest. We then extend it to achieve the first grammar representation able of extracting text substrings, and of searching the text for patterns, in time o(n). We also give byproducts on representing binary relations.