Constructing LZ78 tries and position heaps in linear time for large alphabets

Abstract

We present the first worst-case linear-time algorithm to compute the Lempel-Ziv 78 factorization of a given string over an integer alphabet. Our algorithm is based on nearest marked ancestor queries on the suffix tree of the given string. We also show that the same technique can be used to construct the position heap of a set of strings in worst-case linear time, when the set of strings is given as a trie.

Lempel-Ziv Computation in Compressed Space (LZ-CICS)

“…Since LZ78 factors are naturally represented in a trie, the so-called LZ trie, improving LZ78 computation can be done, among others, by using sophisticated trie implementations [9,10], or by superimposing the suffix tree with the suffix trie [11,12]. We follow the latter approach.…”

Non-Overlapping LZ77 Factorization and LZ78 Substring Compression Queries with Suffix Trees

“…Here, we consider answering substring compression queries with the LZ78 factorization (which is actually also an SLP ( [44], to finally apply the aforementioned algorithm of Bannai et al [43]. The fastest LZ78 factorization algorithms [4,45] can answer a LZ78 substring compression query in O(|I|) time alphabet independently. For small alphabet sizes, the running time O(|I|(lg lg |I|) 2 /(log σ |I| lg lg lg |I|)) of the LZ78 factorization algorithm of Jansson et al [46] becomes even sub-linear in |I|.…”

“…To obtain similar bounds for LZ78, we could adapt the approach of Bille et al [36] to preprocess the LZ78 factorization of all suffixes of T, but that would give us a data structure with super-linear preprocessing time (and possibly super-linear space). Here, we borrow the idea from Nakashima et al [45] to superimpose the suffix tree with the LZ78 trie, and use a data structure for answering nearest marked ancestor queries to find the lowest marked suffix tree node on the path from the root to a leaf. This data structure [47] takes O(n lg n) bits of space, and can answer a nearest marked ancestor query in O(1) amortized time.…”

“…Figure 7 gives another visualization of the same example with the LZ trie, which represents each factor as a node (the root represents the factor F 0 ). The node representing the factor F y has a child representing the factor F x connected with an edge labeled by a character c ∈ Σ if and only if F x = F y c. An observation of Nakashima et al [45] (Section 3) is that the LZ trie is a connected subgraph of the suffix trie containing its root. We can therefore simulate the LZ trie by marking nodes in the suffix trie.…”

Practical Evaluation of Lempel-Ziv-78 and Lempel-Ziv-Welch Tries

“…] and in practice [14, 13, e.g. ], only recent interest in LZ78 can be observed: just in 2015 Nakashima et al [26] gave the first (theoretical) linear time algorithm for LZ78. On the practical side, we are not aware of any systematic study.…”