Induced Sorting Suffixes in External Memory with Better Design and Less Space

Liu, Wei Jun; Nong, Ge; Chan, Wai Hong; Wu, Yi

doi:10.1007/978-3-319-23826-5_9

Cited by 11 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These algorithms cannot be used with large datasets, as they require much more memory than the sequences themselves. Suffix arrays can be built on disk [9], but even the fastest algorithms cannot index the data faster than 1-2 Mbp/s [10,11,12,13,14,15].…”

Section: Introductionmentioning

confidence: 99%

Burrows-Wheeler Transform for Terabases

Sirén

2016

2016 Data Compression Conference (DCC)

View full text Add to dashboard Cite

In order to avoid the reference bias introduced by mapping reads to a reference genome, bioinformaticians are investigating reference-free methods for analyzing sequenced genomes. With large projects sequencing thousands of individuals, this raises the need for tools capable of handling terabases of sequence data. A key method is the Burrows-Wheeler transform (BWT), which is widely used for compressing and indexing reads. We propose a practical algorithm for building the BWT of a large read collection by merging the BWTs of subcollections. With our 2.4 Tbp datasets, the algorithm can merge 600 Gbp/day on a single system, using 30 gigabytes of memory overhead on top of the run-length encoded BWTs.

show abstract

Section: Introductionmentioning

confidence: 99%

Burrows-Wheeler Transform for Terabases

Sirén

2016

2016 Data Compression Conference (DCC)

View full text Add to dashboard Cite

show abstract

“…As future work one can modify algorithms for single strings to handle string collections, such as the extended versions of SAIS that construct the suffix array in external memory (e.g. [12,81,58,45]), as well as the modified versions of SAIS that compute directly the Burrows-Wheeler transform [85] and the Φ array [35]. Moreover one could improve the practical performance of our algorithms by interleaving the storage of the arrays SA, LCP and DA in order to reduce the effect of access to random memory positions during the suffix induction.…”

Section: Methodsmentioning

confidence: 99%

“…There also exist construction alternatives for external memory (e.g. [7,40,44,58,12,16]) and parallel architectures (e.g. [86,59,57,101,53]).…”

Section: Related Workmentioning

confidence: 99%

“…As future work one can modify algorithms for single strings to handle string collections (e.g. [12,81,58,45,85,35]). Moreover one could improve the practical performance of our algorithms by interleaving the storage of the suffix array with the LCP and document arrays, reducing the effect of random memory accesses during the suffix induction.…”

Section: Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

Engineering augmented suffix sorting algorithms

Louza¹

View full text Add to dashboard Cite

Nesta tese estudamos problemas relacionados com a ordenação de sufixos e a construção de estruturas de dados que desempenham um papel fundamental em indexação de textos e compressão de dados. Esta tese contribui com novos algoritmos para a construção do vetor de sufixos, da transformada de Burrows-Wheeler (BWT) e do vetor de prefixo comum mais longo (LCP). Esta tese é organizada como uma coletânea de artigos publicados em periódicos peer-reviewed. Nossa primeira contribuição é um algoritmo in-place que calcula a BWT e o vetor LCP simultaneamente em tempo quadrático. Nossa segunda contribuição é um algoritmo de ordenação de sufixos que constrói o vetor de sufixos e o vetor LCP em tempo e espaço ótimos para cadeias de alfabetos de tamanho constante. Nossa terceira contribuição é um conjunto de algoritmos para a construção do vetor de sufixos aumentado com o vetor LCP e com o vetor de documentos para coleções de cadeias. As soluções apresentadas nesta tese contribuem com melhorias teóricas e avanços práticos na construção de importantes estruturas de dados para processamento de cadeias.

show abstract