Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisons

Besta, Maciej; Raghavendra, Kanakagiri,; Mustafa, Harun; Karasikov, Mikhail; Rätsch, Gunnar; Hoefler, Torsten; Solomonik, Edgar

doi:10.48550/arxiv.1911.04200

Cited by 3 publications

(3 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These works target popular graph algorithms such as BFS or PageRank. Multiplication of matrices and vectors [26,87] has also been addressed in the context of FPGAs [55,56,92,126,134,151]; these efforts could be used for energy-efficient and high-performance graph analytics on FPGAs due to the possibility of expressing graph algorithms in the language of linear algebra [82]. Our work differs from these designs as we focus on the problem of finding graph matchings.…”

Section: Graph Processing On Fpgasmentioning

confidence: 99%

Substream-Centric Maximum Matchings on FPGA

Besta

Fischer

Ben-Nun

et al. 2019

Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

Self Cite

View full text Add to dashboard Cite

Developing high-performance and energy-efficient algorithms for maximum matchings is becoming increasingly important in social network analysis, computational sciences, scheduling, and others. In this work, we propose the first maximum matching algorithm designed for FPGAs; it is energy-efficient and has provable guarantees on accuracy, performance, and storage utilization. To achieve this, we forego popular graph processing paradigms, such as vertex-centric programming, that often entail large communication costs. Instead, we propose a substream-centric approach, in which the input stream of data is divided into substreams processed independently to enable more parallelism while lowering communication costs. We base our work on the theory of streaming graph algorithms and analyze 14 models and 28 algorithms. We use this analysis to provide theoretical underpinning that matches the physical constraints of FPGA platforms. Our algorithm delivers high performance (more than 4× speedup over tuned parallel CPU variants), low memory, high accuracy, and effective usage of FPGA resources. The substream-centric approach could easily be extended to other algorithms to offer low-power and high-performance graph processing on FPGAs.

show abstract

Section: Graph Processing On Fpgasmentioning

confidence: 99%

Substream-Centric Maximum Matchings on FPGA

Besta

Fischer

Ben-Nun

et al. 2019

Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

Self Cite

View full text Add to dashboard Cite

show abstract

“…One could also investigate topology-aware or routing-aware data distribution for graph streaming, especially together with recent high-performance network topologies [29], [130] and routing [37], [144], [88]. Finally, ensuring speedups due to different data modeling abstractions, such as the algebraic abstraction [126], [33], [34], [136], may be a promising direction.…”

Section: Challengesmentioning

confidence: 99%

Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and Systems

Besta¹,

Fischer²,

Kalavri³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Graph processing has become an important part of various areas of computing, including machine learning, medical applications, social network analysis, computational sciences, and others. A growing amount of the associated graph processing workloads are dynamic, with millions of edges added or removed per second. Graph streaming frameworks are specifically crafted to enable the processing of such highly dynamic workloads. Recent years have seen the development of many such frameworks. However, they differ in their general architectures (with key details such as the support for the parallel execution of graph updates, or the incorporated graph data organization), the types of updates and workloads allowed, and many others. To facilitate the understanding of this growing field, we provide the first analysis and taxonomy of dynamic and streaming graph processing. We focus on identifying the fundamental system designs and on understanding their support for concurrency and parallelism, and for different graph updates as well as analytics workloads. We also crystallize the meaning of different concepts associated with streaming graph processing, such as dynamic, temporal, online, and time-evolving graphs, edge-centric processing, models for the maintenance of updates, and graph databases. Moreover, we provide a bridge with the very rich landscape of graph streaming theory by giving a broad overview of recent theoretical related advances, and by analyzing which graph streaming models and settings could be helpful in developing more powerful streaming frameworks and designs. We also outline graph streaming workloads and research challenges.

show abstract

“…Large graphs are a basis of many problems in machine learning, medicine, social network analysis, computational sciences, and others [15,25,106]. The growing graph sizes, reaching one trillion edges in 2015 (the Facebook social graph [48]) and 12 trillion edges in 2018 (the Sogou webgraph [101]), require unprecedented amounts of compute power, storage, and energy.…”

Section: Introductionmentioning

confidence: 99%

Slim Graph: Practical Lossy Graph Compression for Approximate Graph Processing, Storage, and Analytics

Besta,

Weber,

Gianinazzi

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We propose Slim Graph: the first programming model and framework for practical lossy graph compression that facilitates high-performance approximate graph processing, storage, and analytics. Slim Graph enables the developer to express numerous compression schemes using small and programmable compression kernels that can access and modify local parts of input graphs. Such kernels are executed in parallel by the underlying engine, isolating developers from complexities of parallel programming. Our kernels implement novel graph compression schemes that preserve numerous graph properties, for example connected components, minimum spanning trees, or graph spectra. Finally, Slim Graph uses statistical divergences and other metrics to analyze the accuracy of lossy graph compression. We illustrate both theoretically and empirically that Slim Graph accelerates numerous graph algorithms, reduces storage used by graph datasets, and ensures high accuracy of results. Slim Graph may become the common ground for developing, executing, and analyzing emerging lossy graph compression schemes.

show abstract

Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisons

Cited by 3 publications

References 64 publications

Substream-Centric Maximum Matchings on FPGA

Substream-Centric Maximum Matchings on FPGA

Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and Systems

Slim Graph: Practical Lossy Graph Compression for Approximate Graph Processing, Storage, and Analytics

Contact Info

Product

Resources

About