The Myriad Virtues of Subword Trees

Apostolico, Alberto

doi:10.1007/978-3-642-82456-2_6

Cited by 217 publications

(183 citation statements)

References 16 publications

Supporting

Mentioning

183

Contrasting

Order By: Relevance

“…This is the case, for instance, of suffix SA 0 [2] in Figure 2(a), which is not in T 0 but in T 1 . This will have implications when the suffix array search algorithm needs to access the text suffixes.…”

Section: Global Suffix Arraysmentioning

confidence: 93%

“…Suffix trees are probably the most important data structures for processing strings [2,29]. They are extensively used not only for full-text searching, but also for more complex pattern matching and discovery problems.…”

Section: The Importance Of Suffix Arrays In Text Processing and Searcmentioning

confidence: 99%

“…Notice how each bit vector marks with a 1 the global positions that are indexed at each processor. For instance, in our example we have that, in processor P 3 , SA 3 [2] corresponds to the global entry SA [9] (as the 2nd 1 in the bit vector is B 3 [9]). Conversely, given the global position SA [9], we know that processor P 3 stores it, as B 3 [9] = 1.…”

Section: Global Suffix Array With Local Textmentioning

confidence: 99%

See 2 more Smart Citations

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

View full text Add to dashboard Cite

Text search is a classical problem in Computer Science, with many data-intensive applications. For this problem, suffix arrays are among the most widely known and used data structures, enabling fast searches for phrases, terms, substrings and regular expressions in large texts. Potential application domains for these operations include large-scale search services, such as Web search engines, where it is necessary to efficiently process intensive-traffic streams of on-line queries. This paper proposes strategies to enable such services by means of suffix arrays. We introduce techniques for deploying suffix arrays on clusters of distributed-memory processors and then study the processing of multiple queries on the distributed data structure. Even though the cost of individual search operations in sequential (non-distributed) suffix arrays is low in practice, the problem of processing multiple queries on distributed-memory systems, so that hardware resources are used efficiently, is relevant to services aimed at achieving high query throughput at low operational costs. Our theoretical and experimental performance studies show that our proposals are suitable solutions for building efficient and scalable on-line search services based on suffix arrays. IntroductionIn the last decade, the design of efficient data structures and algorithms for textual databases and related applications has received a great deal of attention, due to the rapid growth of the amount of text data available from different sources. Typical applications support text searches over big text collections in a client-server fashion, where the user queries are answered by a dedicated server [15]. The server efficiency-in terms of running time-is of paramount importance in cases where the services demanded by clients generate a heavy work load. A feasible way to overcome the limitations of sequential computers is to resort to the use of several computers, or processors, which work together to serve the ever increasing client demands [19].One such approach to efficient parallelization is to distribute the data onto the processors, in such a way that it becomes feasible to exploit locality via parallel processing of user requests, each on a subset of the data. As opposed to shared-memory models, this distributed-memory model provides the benefit of better * Corresponding author. Address: Av. España 1680, Valparaíso, Chile. Phone: +56 2 432 6722. Fax: +56 2 432 6702. , in distributed memory systems, and describes strategies to reduce the inter-processor communication and to improve the load balance at search time. Indexed Text SearchingThe advent of powerful processors and cheap storage has enabled alternative models for information retrieval, other than the traditional one of a collection of documents indexed by a fixed set of keywords. One is the full text model, in which the user expresses its information need via words, phrases or patterns to be matched for, and the information system retrieves those documents containing the user-specified pa...

show abstract

Section: Global Suffix Arraysmentioning

confidence: 93%

Section: The Importance Of Suffix Arrays In Text Processing and Searcmentioning

confidence: 99%

Section: Global Suffix Array With Local Textmentioning

confidence: 99%

See 1 more Smart Citation

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

View full text Add to dashboard Cite

show abstract

“…Muthukrishnan defined the array E[1, n] such that E[i] is the document containing the starting position of the lexicographically ith suffix in T . If we store a suffix tree [38,1] for T then, given a pattern, we can quickly find the lexicographic ranks i and j of the first and last suffixes starting with the pattern. This is equivalent to finding the range A[i..j] in the suffix array [27] A for T that lists the starting positions of all the suffixes of T that start with the pattern.…”

Section: Listingmentioning

confidence: 99%

Colored Range Queries and Document Retrieval

Gagie

Navarro

Puglisi

2010

String Processing and Information Retrieval

View full text Add to dashboard Cite

Abstract. Colored range queries are a well-studied topic in computational geometry and database research that, in the past decade, have found exciting applications in information retrieval. In this paper we give improved time and space bounds for three important one-dimensional colored range queries -colored range listing, colored range top-k queries and colored range counting -and, thus, new bounds for various document retrieval problems on general collections of sequences. Specifically, we first describe a framework including almost all recent results on colored range listing and document listing, which suggests new combinations of data structures for these problems. For example, we give the fastest compressed data structures for colored range listing and document listing, and an efficient data structure for document listing whose size is bounded in terms of the high-order entropies of the library of documents. We then show how (approximate) colored top-k queries can be reduced to (approximate) range-mode queries on subsequences, yielding the first efficient data structure for this problem. Finally, we show how a modified wavelet tree can support colored range counting in logarithmic time and space that is succinct whenever the number of colors is superpolylogarithmic in the length of the sequence.

show abstract

“…Myriad non-trivial combinatorial questions concerning strings turn out to have efficient solutions via extensive use of suffix trees [Apostolico 1985]. As a theoretical tool, suffix trees have a fundamental role in plethora of algorithmic results in the area of string matching and sequence analysis.…”

Section: Introductionmentioning

confidence: 99%

Engineering a Compressed Suffix Tree Implementation

Välimäki

Gerlach

Dixit

et al. 2007

Experimental Algorithms

View full text Add to dashboard Cite

Suffix tree is one of the most important data structures in string algorithms and biological sequence analysis. Unfortunately, when it comes to implementing those algorithms and applying them to real genomic sequences, often the main memory size becomes the bottleneck. This is easily explained by the fact that while a DNA sequence of length n from alphabet Σ = {A, C, G, T } can be stored in n log |Σ| = 2n bits, its suffix tree occupies O(n log n) bits. In practice, the size difference easily reaches factor 50.We report on an implementation of the compressed suffix tree very recently proposed by Sadakane (Theory of Computing Systems, in press). The compressed suffix tree occupies space proportional to the text size, i.e. O(n log |Σ|) bits, and supports all typical suffix tree operations with at most log n factor slowdown. Our experiments show that, e.g. on a 10 MB DNA sequence, the compressed suffix tree takes 10% of the space of the normal suffix tree. At the same time, a representative algorithm is slowed down by factor 30.Our implementation follows the original proposal in spirit, but some internal parts are tailored towards practical implementation. Our construction algorithm has time requirement O(n log n log |Σ|) and uses closely the same space as the final structure while constructing it: on the 10 MB DNA sequence, the maximum space usage during construction is only 1.5 times the final product size. As by-products, we develop a method to create Succinct Suffix Array directly from Burrows-Wheeler transform and a space-efficient version of suffixes-insertion algorithm to build balanced parentheses representation of suffix tree from LCP information.

show abstract

The Myriad Virtues of Subword Trees

Cited by 217 publications

References 16 publications

Distributed text search using suffix arrays

Distributed text search using suffix arrays

Colored Range Queries and Document Retrieval

Engineering a Compressed Suffix Tree Implementation

Contact Info

Product

Resources

About