Induced Sorting Suffixes in External Memory

Nong, Ge; Chan, Wai Hong; Hu, Sheng; Wu, Yi

doi:10.1145/2699665

Cited by 27 publications

(12 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in the experiments reported in [18], DSA-IS is only ever "marginally faster" than eSAIS, which is also confirmed in our experiments.…”

Section: Comparison Of Algorithms Based Onsupporting

confidence: 90%

“…When we need to know X[i − 1], we determine which block contains the position i − 1 and then read the next symbol from the precomputed sequence associated with that block. A more cumbersome form of blockwise preinducing was introduced in [18].…”

Section: Blockwise Preinducingmentioning

confidence: 99%

“…We also demonstrate the efficacy of our implementation for handling strings on large alphabets (with many millions of distinct symbols), which is important, e.g., for applications in natural language processing [21] and information retrieval [10]. Efficiency for large alphabets has been to date unaddressed by previous studies on EM suffix sorting [6,7,4,12,18,13], in all of which a byte alphabet is assumed. While eSAIS could probably be modified for large alphabets fairly easily, with pSAscan this would require a major redesign.…”

Section: Introductionmentioning

confidence: 98%

“…A serious limiting factor of eSAIS is high disk usage: about 24 times the size of the input. Another disk-based implementation of induced sorting DSA-IS [18] is slower than eSAIS and uses even more disk space in our experiments.…”

Section: Introductionmentioning

confidence: 99%

“…al. [18]. 5 It uses a technique similar to our preinducing, but the algorithm and implementation assume that n = O(M 2 /B), and thus the algorithm is less scalable than eSAIS.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Engineering External Memory Induced Suffix Sorting

Kärkkäinen

Kempa

Puglisi

et al. 2017

2017 Proceedings of the Ninteenth Workshop on Algorithm Engineering and Experiments (ALENEX)

View full text Add to dashboard Cite

Suffix sorting -determining the lexicographical order of all the suffixes of a string -is one of the most important problems in string processing. The resulting data structure is called the suffix array (SA) and underpins dozens of applications in bioinformatics, data compression, and information retrieval. When the size of the input string or the SA exceeds that of internal memory (RAM), an external memory (EM) suffix sorting algorithm must be used. The most scalable of these EM methods is due to Bingmann et al. (Proc. ALENEX 2013), and is essentially a careful disk-based implementation of the so-called induced sorting technique used by the fastest RAM suffix sorting algorithms.In this paper we show how to greatly improve the efficiency of induced suffix sorting in external memory via a non-trivial reorganization of the computation involved. Our experiments show this new approach to be twice as fast as state-of-the-art methods, while, just as significantly, using a third of the disk memory. We also demonstrate the efficacy of our implementation for handling strings on large alphabets (with many millions of distinct symbols), which is important, e.g., for applications in natural language processing and information retrieval, but unaddressed by previous EM suffix sorting implementations.Our implementation uses a (EM) radix heap data structure and, as a side result of independent interest, we introduce a new operation for radix heaps and other monotone priority queues called min-comp, which we believe to be useful for many other applications, including discrete event simulation and sweep line algorithms, even in internal memory.

show abstract

“…However, in the experiments reported in [18], DSA-IS is only ever "marginally faster" than eSAIS, which is also confirmed in our experiments.…”

Section: Comparison Of Algorithms Based Onsupporting

confidence: 90%

Section: Blockwise Preinducingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

“…al. [18]. 5 It uses a technique similar to our preinducing, but the algorithm and implementation assume that n = O(M 2 /B), and thus the algorithm is less scalable than eSAIS.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Engineering External Memory Induced Suffix Sorting

Kärkkäinen

Kempa

Puglisi

et al. 2017

2017 Proceedings of the Ninteenth Workshop on Algorithm Engineering and Experiments (ALENEX)

View full text Add to dashboard Cite

show abstract

Optimal In-Place Suffix Sorting

Huo

2018

String Processing and Information Retrieval

View full text Add to dashboard Cite

The suffix array is a fundamental data structure for many applications that involve string searching and data compression. Designing time/space-efficient suffix array construction algorithms has attracted significant attention and considerable advances have been made for the past 20 years. We obtain the first in-place suffix array construction algorithms that are optimal both in time and space for (read-only) integer alphabets. Concretely, we make the following contributions:1. For integer alphabets, we obtain the first suffix sorting algorithm which takes linear time and uses only O(1) workspace (the workspace is the total space needed beyond the input string and the output suffix array). The input string may be modified during the execution of the algorithm, but should be restored upon termination of the algorithm.2. We strengthen the first result by providing the first in-place linear time algorithm for read-only integer alphabets with |Σ| = O(n) (i.e., the input string cannot be modified). This algorithm settles the open problem posed by Franceschini and Muthukrishnan in ICALP 2007. The open problem asked to design in-place algorithms in o(n log n) time and ultimately, in O(n) time for (read-only) integer alphabets with |Σ| ≤ n. Our result is in fact slightly stronger since we allow |Σ| = O(n). 3. Besides, for the read-only general alphabets (i.e., only comparisons are allowed), we present an optimal in-place O(n log n) time suffix sorting algorithm, recovering the result obtained by Franceschini and Muthukrishnan which was an open problem posed by Manzini and Ferragina in ESA 2002.

show abstract

SACABench: Benchmarking Suffix Array Construction

Bahne

Bertram

Böcker

et al. 2019

String Processing and Information Retrieval

View full text Add to dashboard Cite

We present a practical comparison of suffix array construction algorithms on modern hardware. The benchmark is conducted using our new benchmark framework SACABench, which allows for an easy deployment of publicly available implementations, simple plotting of the results, and straight forward support to include new construction algorithms. We use the framework to develop a construction algorithm running on the GPU that is competitive with the fastest parallel algorithm in our test environment.

show abstract

Induced Sorting Suffixes in External Memory

Cited by 27 publications

References 16 publications

Engineering External Memory Induced Suffix Sorting

Engineering External Memory Induced Suffix Sorting

Optimal In-Place Suffix Sorting

SACABench: Benchmarking Suffix Array Construction

Contact Info

Product

Resources

About