LZ trie and dictionary compression

Abstract

An efficient algorithm for trie compression has already been described. Here we present its practical value and demonstrate its superiority in terms of space savings to other methods of lexicon compression. Apart from simple lexicons, a compressed trie can, with some additional processing, be used as a component in the compact representation of simple static databases. We present the potential of the algorithm in compressing natural language dictionaries.

Fast construction of space‐optimized recursive automaton

“…The best compression is obtained with the LZ trie [8]. A comparison with other recursive approaches can be found in [12,19], and extensive experimentation that is reported in [5] confirms that the LZ trie method produces the smallest compressed automaton in all cases.…”

“…The included sets cover a sufficiently wide range of natural language applications. All of the sets are publicly available and have been used previously as the benchmarks for automata compression: dimacs, random and word lists in [5,8,20], and wikipedia2 and pol_morph in [5]. Daciuk et al [5] provide a link to all of the data sets ¶ , and we invite readers to try our software on them.…”

“…The recursive call then replaces the identical substructures. This process is the crux of the methods proposed in . One of the earlier mentions of subautomata substitution can be found in , although this paper addresses the different context of push‐down automata.…”

“…The best result so far in automata compression, regarding only the compression factor, is the LZ trie , which also exploits the RA concept. The problem with the LZ trie was a long construction time for large inputs and a complicated procedure with no available software.…”

Zgoščena predstavitev slovarjev izgovarjav s končnimi super pretvorniki

“…V literaturi je mogoče zaslediti tri osnovne pristope, ki omogočajo zgoščeno predstavitev slovarjev izgovarjav, in sicer z uporabo oštevilčenih končnih avtomatov (Lucchesi in Kowaltowski, 1993;Daciuk in Piskorski, 2011), dreves predpon (Ristov, 2005) ter z uporabo končnih pretvornikov (Mohri, 1994;Golob at al., 2012). V članku bomo predstavili nov način zgoščene predstavitve slovarjev izgovarjav z uporabo t. i. končnih super pretvornikov, ki predstavljajo nadgradnjo običajnih končnih pretvornikov.…”