The Smallest Grammar Problem Revisited

Hucke, Danny; Lohrey, Markus; Reh, Carl Philipp

doi:10.1007/978-3-319-46049-9_4

Cited by 18 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Those papers exhibit O(log n)-approximations to the smallest grammar, as well as several others [32,17,18]. A negative result about the approximation are string families where g = Ω(z no log n/ log log n) [6,15] and, recently, g rl = Ω(z no log n/ log log n) [3].…”

Section: Grammar Compressionmentioning

confidence: 99%

On the Approximation Ratio of Lempel-Ziv Parsing

Gagie

Navarro

Prezza

2018

LATIN 2018: Theoretical Informatics

View full text Add to dashboard Cite

Shannon's entropy is a clear lower bound for statistical compression. The situation is not so well understood for dictionary-based compression. A plausible lower bound is b, the least number of phrases of a general bidirectional parse of a text, where phrases can be copied from anywhere else in the text. Since computing b is NP-complete, a popular gold standard is z, the number of phrases in the Lempel-Ziv parse of the text, where phrases can be copied only from the left. While z can be computed in linear time, almost nothing has been known for decades about its approximation ratio with respect to b. In this paper we prove that z = O(b log(n/b)), where n is the text length. We also show that the bound is tight as a function of n, by exhibiting a string family where z = Ω(b log n). Our upper bound is obtained by building a run-length context-free grammar based on a locally consistent parsing of the text. Our lower bound is obtained by relating b with r, the number of equal-letter runs in the Burrows-Wheeler transform of the text. On our way, we prove other relevant bounds between compressibility measures.

show abstract

Section: Grammar Compressionmentioning

confidence: 99%

On the Approximation Ratio of Lempel-Ziv Parsing

Gagie

Navarro

Prezza

2018

LATIN 2018: Theoretical Informatics

View full text Add to dashboard Cite

show abstract

“…There are also several lower bounds on further possible upper bounds, for example, there are text families for which g = Ω(g rl log n) and z no = Ω(z log n) (i.e., T = a n−1 $); g = Ω(z no log n/ log log n) [47]; e ≥ m = Ω(max(r, z) · n) [6] and thus e = Ω(g · n/ log n) since g = O(z log n); min(r, z) = Ω(m · n) [6]; r = Ω(z no log n) [6,84]; z = Ω(r log n) [84]; r = Ω(g log n/ log log n) (since on a g A Failure in the Analysis based on RePair and Relatives…”

Section: Map Of the Relations Between Repetitiveness Measuresmentioning

confidence: 99%

Optimal-Time Text Indexing in BWT-runs Bounded Space

Gagie¹,

Navarro²,

Prezza³

2018

Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms

126

View full text Add to dashboard Cite

Indexing highly repetitive texts -such as genomic databases, software repositories and versioned text collections -has become an important problem since the turn of the millennium. A relevant compressibility measure for repetitive texts is r, the number of runs in their Burrows-Wheeler Transform (BWT). One of the earliest indexes for repetitive collections, the Run-Length FMindex, used O(r) space and was able to efficiently count the number of occurrences of a pattern of length m in the text (in loglogarithmic time per pattern symbol, with current techniques). However, it was unable to locate the positions of those occurrences efficiently within a space bounded in terms of r. Since then, a number of other indexes with space bounded by other measures of repetitiveness -the number of phrases in the LempelZiv parse, the size of the smallest grammar generating the text, the size of the smallest automaton recognizing the text factors -have been proposed for efficiently locating, but not directly counting, the occurrences of a pattern. In this paper we close this long-standing problem, showing how to extend the Run-Length FMindex so that it can locate the occ occurrences efficiently within O(r) space (in loglogarithmic time each), and reaching optimal time O(m + occ) within O(r log(n/r)) space, on a RAM machine with words of w = Ω(log n) bits. Raising the space to O(rw log σ (n/r)), we support locate in O(m log(σ)/w + occ) time, which is optimal in the packed setting and had not been obtained before in compressed space. We also describe a structure using O(r log(n/r)) space that replaces the text and efficiently extracts any text substring, with an O(log(n/r)) additive time penalty over the optimum. Preliminary experiments show that our new structure outperforms the alternatives by orders of magnitude in the space/time tradeoff map.

show abstract

“…Instead, grammar compression allows extracting any text symbol in logarithmic time using O(r lg N ) bits, where r is the size of the grammar [8,54]. It is possible to obtain a grammar of size r = O(z lg(N/z)) [10,30], which using standard methods [50] can be tweaked to r = n/ lg σ N + s lg N under our repetitiveness model. Thus the space we might aim at for indexing is O(n lg σ + s lg 2 N ) bits.…”

Section: Modeling Repetitivenessmentioning

confidence: 99%

Document listing on repetitive collections with guaranteed performance

Navarro

2019

Theoretical Computer Science

View full text Add to dashboard Cite

We consider document listing on string collections, that is, finding in which strings a given pattern appears. In particular, we focus on repetitive collections: a collection of size N over alphabet [1, σ] is composed of D copies of a string of size n, and s edits are applied on ranges of copies. We introduce the first document listing index with sizeÕ(n + s), precisely O((n lg σ + s lg 2 N ) lg D) bits, and with useful worst-case time guarantees: Given a pattern of length m, the index reports the ndoc > 0 strings where it appears in time O(m lg 1+ N · ndoc), for any constant > 0 (and tells in time O(m lg N ) if ndoc = 0). Our technique is to augment a range data structure that is commonly used on grammar-based indexes, so that instead of retrieving all the pattern occurrences, it computes useful summaries on them. We show that the idea has independent interest: we introduce the first grammar-based index that, on a text T [1, N ] with a grammar of size r, uses O(r lg N ) bits and counts the number of occurrences of a pattern P [1, m] in time O(m 2 + m lg 2+ r), for any constant > 0. We also give the first index using O(z lg(N/z) lg N ) bits, where T is parsed by Lempel-Ziv into z phrases, counting occurrences in time O(m lg 2+ N ).

show abstract

The Smallest Grammar Problem Revisited

Cited by 18 publications

References 15 publications

On the Approximation Ratio of Lempel-Ziv Parsing

On the Approximation Ratio of Lempel-Ziv Parsing

Optimal-Time Text Indexing in BWT-runs Bounded Space

Document listing on repetitive collections with guaranteed performance

Contact Info

Product

Resources

About