Cloud-based parallel suffix array construction based on MPI

Abdelhadi, Ahmed; Kandil, Ahmed H.; Abouelhoda, Mohamed

doi:10.1109/mecbme.2014.6783271

Cited by 7 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We measure the performance of the parallel BWT and FM index backward_search based on our fast hybrid skew/prefix-doubling implementation on two representative real-world datasets "enwik8" (first 10 Much of the interesting work in GPU computing has been the result of brute-force techniques, judiciously applied. Often, GPU computing practitioners have found that the loss of efficiency by using brute force is more than offset by the performance advantages of the GPU.…”

Section: Millions Of Chars Per Secondmentioning

confidence: 99%

“…Together these issues make FAK a poor match for GPUs, but for more coarse‐grained parallel machines, it is a potentially better fit: for instance, Abdelhadi et al . recently described an MPI‐based implementation of the FAK algorithm on cloud‐based computer clusters.…”

Section: Related Workmentioning

confidence: 99%

“…Finally, it allocates a different amount of work to each bucket, leading to load imbalance across processors. Together these issues make FAK a poor match for GPUs, but for more coarse-grained parallel machines, it is a potentially better fit: for instance, Abdelhadi et al [10] recently described an MPI-based implementation of the FAK algorithm on cloud-based computer clusters. Kulla and Sanders [11] proposed another parallel distributed-memory SACA, based on the lineartime skew approach of Kärkkäinen and Sanders [4].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fast parallel skew and prefix‐doubling suffix array construction on the GPU

Wang

Baxter

Owens

2016

Concurrency and Computation

View full text Add to dashboard Cite

Summary Suffix arrays are fundamental full‐text index data structures of importance to a broad spectrum of applications in such fields as bioinformatics, Burrows–Wheeler transform‐based lossless data compression, and information retrieval. In this work, we propose and implement two massively parallel approaches on the graphics processing unit (GPU) based on two classes of suffix array construction algorithms. The first, parallel skew, makes algorithmic improvements to the previous work of Deo and Keely to achieve a speedup of 1.45x over their work. The second, a hybrid skew and prefix‐doubling implementation, is the first of its kind on the GPU and achieves a speedup of 2.3–4.4x over Osipov's prefix‐doubling and 2.4–7.9x over our skew implementation on large datasets. Our implementations rely on two efficient parallel primitives, a merge and a segmented sort. We theoretically analyze the two formulations of suffix array construction algorithms and show performance comparisons on a large variety of practical inputs. We conclude that, with the novel use of our efficient segmented sort, prefix‐doubling is more competitive than skew on the GPU. We also demonstrate the effectiveness of our methods in our implementations of the Burrows‐Wheeler transform and in a parallel full‐text, minute‐space‐index for pattern searching. Copyright © 2016 John Wiley & Sons, Ltd.

show abstract

Section: Millions Of Chars Per Secondmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast parallel skew and prefix‐doubling suffix array construction on the GPU

Wang

Baxter

Owens

2016

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Drawbacks of this approach are the worst-case run-time of O(n 2 ), and the fact that the input string needs to be in-memory on each processor. A more recent work by Abdelhadi et al [1] provides a MPI based implementation of the FAK algorithm, adapted to cloud computing on AWS (Amazon Web Services).…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, we run the shared-memory parallel suffix array and LCP construction tool mkESA [13]. Additionally, we directly compare our implementation with the FAK implementation cloudSACA by Abdelhadi et al [1]. We ran all aforementioned suffix array construction algorithms on the same hardware described earlier, thus allowing direct comparison.…”

Section: Comparison With Prior State-of-the-artmentioning

confidence: 99%

Parallel distributed memory construction of suffix and longest common prefix arrays

Flick

Aluru

2015

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Suffix arrays and trees are fundamental string data structures of importance to many applications in computational biology. Consequently, their parallel construction is an actively studied problem. To date, algorithms with best practical performance lack efficient worst-case run-time guarantees, and vice versa. In addition, much of the recent work targeted low core count, shared memory parallelization. In this paper, we present parallel algorithms for distributed memory construction of suffix arrays and longest common prefix (LCP) arrays that simultaneously achieve good worstcase run-time bounds and superior practical performance. Our algorithms run in O(Tsort(n, p) • log n) worst-case time where Tsort(n, p) is the run-time of parallel sorting. We present several algorithm engineering techniques that improve performance in practice. We demonstrate the construction of suffix and LCP arrays of the human genome in less than 8 seconds on 1,024 Intel Xeon cores, reaching speedups of over 110X compared to the best sequential suffix array construction implementation divsufsort.

show abstract

SACABench: Benchmarking Suffix Array Construction

Bahne

Bertram

Böcker

et al. 2019

String Processing and Information Retrieval

View full text Add to dashboard Cite

We present a practical comparison of suffix array construction algorithms on modern hardware. The benchmark is conducted using our new benchmark framework SACABench, which allows for an easy deployment of publicly available implementations, simple plotting of the results, and straight forward support to include new construction algorithms. We use the framework to develop a construction algorithm running on the GPU that is competitive with the fastest parallel algorithm in our test environment.

show abstract

Cloud-based parallel suffix array construction based on MPI

Cited by 7 publications

References 6 publications

Fast parallel skew and prefix‐doubling suffix array construction on the GPU

Fast parallel skew and prefix‐doubling suffix array construction on the GPU

Parallel distributed memory construction of suffix and longest common prefix arrays

SACABench: Benchmarking Suffix Array Construction

Contact Info

Product

Resources

About