An incomplex algorithm for fast suffix array construction

Schürmann, Klaus-Bernd; Stoye, Jens

doi:10.1002/spe.768

Cited by 32 publications

(29 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also compared to the enhanced suffix array ESA [Abouelhoda et al 2004]; we used the implementation that is plugged into the Vmatch software package 6 . For suffix array construction, we used the bpr algorithm [Schürmann and Stoye 2005] that is the currently the fastest construction algorithm in practice.…”

Section: Resultsmentioning

confidence: 99%

Engineering a Compressed Suffix Tree Implementation

Välimäki

Gerlach

Dixit

et al. 2007

Experimental Algorithms

View full text Add to dashboard Cite

Suffix tree is one of the most important data structures in string algorithms and biological sequence analysis. Unfortunately, when it comes to implementing those algorithms and applying them to real genomic sequences, often the main memory size becomes the bottleneck. This is easily explained by the fact that while a DNA sequence of length n from alphabet Σ = {A, C, G, T } can be stored in n log |Σ| = 2n bits, its suffix tree occupies O(n log n) bits. In practice, the size difference easily reaches factor 50.We report on an implementation of the compressed suffix tree very recently proposed by Sadakane (Theory of Computing Systems, in press). The compressed suffix tree occupies space proportional to the text size, i.e. O(n log |Σ|) bits, and supports all typical suffix tree operations with at most log n factor slowdown. Our experiments show that, e.g. on a 10 MB DNA sequence, the compressed suffix tree takes 10% of the space of the normal suffix tree. At the same time, a representative algorithm is slowed down by factor 30.Our implementation follows the original proposal in spirit, but some internal parts are tailored towards practical implementation. Our construction algorithm has time requirement O(n log n log |Σ|) and uses closely the same space as the final structure while constructing it: on the 10 MB DNA sequence, the maximum space usage during construction is only 1.5 times the final product size. As by-products, we develop a method to create Succinct Suffix Array directly from Burrows-Wheeler transform and a space-efficient version of suffixes-insertion algorithm to build balanced parentheses representation of suffix tree from LCP information.

show abstract

Section: Resultsmentioning

confidence: 99%

Engineering a Compressed Suffix Tree Implementation

Välimäki

Gerlach

Dixit

et al. 2007

Experimental Algorithms

View full text Add to dashboard Cite

show abstract

“…There are several more sophisticated algorithms, from the original O(n log n) time [Manber and Myers 1993] to the latest O(n) time algorithms Ko and Aluru 2003;Kärkkäinen and Sanders 2003]. In practice, the best current algorithms are not linear-time ones [Larsson and Sadakane 1999;Itoh and Tanaka 1999;Manzini and Ferragina 2004;Schürmann and Stoye 2005]. Fig.…”

Section: Definitionmentioning

confidence: 99%

Compressed full-text indexes

2007

View full text Add to dashboard Cite

Full-text indexes provide fast substring search over large text collections. A serious problem of these indexes has traditionally been their space consumption. A recent trend is to develop indexes that exploit the compressibility of the text, so that their size is a function of the compressed text length. This concept has evolved into self-indexes, which in addition contain enough information to reproduce any text portion, so they replace the text. The exciting possibility of an index that takes space close to that of the compressed text, replaces it, and in addition provides fast search over it, has triggered a wealth of activity and produced surprising results in a very short time, and radically changed the status of this area in less than five years. The most successful indexes nowadays are able to obtain almost optimal space and search time simultaneously.In this paper we present the main concepts underlying self-indexes. We explain the relationship between text entropy and regularities that show up in index structures and permit compressing them. Then we cover the most relevant self-indexes up to date, focusing on the essential aspects on how they exploit the text compressibility and how they solve efficiently various search problems. We aim at giving the theoretical background to understand and follow the developments in this area.

show abstract

“…A suffix tree is a representation of all suffixes of a string in a tree format [13]; it is considered a very powerful data structure [20,21,23]. Suffix trees are heavily used for data mining purposes in many scientific and commercial fields (e.g., financial, marketing, frequent item-sets detection, DNA) due to their advantages in string processing [12,21,22,24].…”

Section: Related Workmentioning

confidence: 99%

“…The most common technique so far is based on a very powerful data structure, the suffix tree [12,13]. Suffix trees are more preferred because they require low space capacity and they lead to very good time complexity that allows fast analysis of time series.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analyzing very large time series using suffix arrays

2014

View full text Add to dashboard Cite

Suffix arrays form a powerful data structure for pattern detection and matching. In a previous work, we presented a novel algorithm (COV) which is the only algorithm that allows the detection of all repeated patterns in a time series by using the actual suffix array. However, the requirements for storing the actual suffix strings even on external media makes the use of suffix arrays impossible for very large time series. We have already proved that using the concept of Longest Expected Repeated Pattern (LERP) allows the actual suffices to be stored in linear capacity O(n) on external media. The repeated pattern detection using LERP has analogous time complexity, and thus makes the analysis of large time series feasible and limited only to the size Further, when empirical knowledge related to the distribution of repeated pattern's length is available, the proposed method (MLERP) can achieve better time performance compared to the standard LERP method and definitely much better than using any other pattern matching algorithm and applying brute force techniques which are unfeasible in logical (human) time frame. Thus, we may argue that MLERP is a very useful tool for detecting all repeated patterns in a time series regardless of its size and hardware limitations.

show abstract

An incomplex algorithm for fast suffix array construction

Cited by 32 publications

References 14 publications

Engineering a Compressed Suffix Tree Implementation

Engineering a Compressed Suffix Tree Implementation

Compressed full-text indexes

Analyzing very large time series using suffix arrays

Contact Info

Product

Resources

About