The Cell Probe Complexity of Succinct Data Structures

Abstract

In the cell probe model with word size 1 (the bit probe model), a static data structure problem is given by a map f : {0,n is a set of possible data to be stored, {0, 1} m is a set of possible queries (for natural problems, we have m n) and f(x, y) is the answer to question y about data x.A solution is given by a representation φ : y). The time t of the query algorithm is the number of bits it reads in φ(x).In this paper, we consider the case of succinct representations where s = n + r for some redundancy r n. For a boolean version of the problem of polynomial evaluation with preprocessing of coefficients, we show a lower bound on the redundancy-query time tradeoff of the form (r + 1)t ≥ Ω(n/ log n).In particular, for very small redundancies r, we get an almost optimal lower bound stating that the query algorithm has to inspect almost the entire data structure (up to a logarithmic factor). We show similar lower bounds for problems satisfying a certain combinatorial property of a coding theoretic flavor. Previously, no ω(m) lower bounds were known on t in the general model for explicit functions, even for very small redundancies.By restricting our attention to systematic or index structures φ satisfying φ(x) = x·φ * (x) for some map φ * (where · denotes concatenation) we show similar lower bounds on the redundancy-query time tradeoff

A Geometric Approach to Lower Bounds for Approximate Near-Neighbor Search and Partial Match

“…space n vs. n 20 ) in the size of the data structure translate to constant multiplicative differences (e.g. log n vs. 20 log n) in the number of bits sent by Alice, making it difficult to prove better bounds for small polynomial, or near-linear data structures (see Gal and Miltersen [14] for a discussion). Specifically for randomized approximate nearest neighbor search, Chakrabarti and Regev [8] show that a common communication complexity technique called 'richness' cannot yield any non-trivial bound.…”

Time-space trade-offs for predecessor search

“…However, a separation for an explicit problem has only been obtained in a very restricted setting. Gál and Miltersen [9] showed such a separation when the space complexity is very close to minimum: given an input of n cells, the space used by the data structure is n + o(n).…”

“…Besides the reduction to communication complexity, and the approach of [9] for very small space, there are no known techniques applicable to static cell-probe complexity with cells of Ω(lg n) bits. In particular, we note that the large body of work initiated by Fredman and Saks [7] only applies to dynamic problems, such as maintaining partial sums or connectivity.…”

Succincter

“…The RANK/SELECT problem has also seen a lot of work in lower bounds [10,16,14,12], particularly bounds applying to "systematic encodings." In this model, the bit vector is stored separately in plain form, and the succinct data structure consists of a sublinear size index on the side.…”

“…Unfortunately, proving this seems beyond the scope of current techniques. The only lower bound for succinct data structures (without the systematic assumption) is via a rather simple idea of Gál and Miltersen [10], which requires that the data structure have an intrinsic error-correcting property. Such a property is not characteristic of our problems.…”