A Fast Algorithm for the Largest Area First Parsing of Real Strings

Abstract

The largest area first parsing of a string often leads to the best results in grammar compression for a variety of input data. However, the fastest existing algorithm has Θ(N 2 log N ) time complexity, which makes it impractical for real-life applications. We present a new largest area first parsing method that has O(N 3 ) complexity in the improbable worst case but works in the quasilinear time for most practical purposes. This result is based on the fact that in the real data, the sum of all depths of an LCP-interval tree, over all of the positions in a suffix array of an input string, is only larger than the size of the input by a small factor α. We present the analysis of the algorithm in terms of α, and the experimental results confirm that our method is practical even for genome sized inputs. We provide the C++11 code for the implementation of our method. Additionally, we show that by a combination of the previous and new algorithms, the worst-case complexity of the largest area first parsing is improved by a factor of 3 √ N .INDEX TERMS Greedy grammar compression, largest area first parsing, dynamic text indexing, enhanced suffix array.