High-Productivity and High-Performance Analysis of Filtered Semantic Graphs

Abstract

Abstract-High performance is a crucial consideration when executing a complex analytic query on a massive semantic graph. In a semantic graph, vertices and edges carry attributes of various types. Analytic queries on semantic graphs typically depend on the values of these attributes; thus, the computation must view the graph through a filter that passes only those individual vertices and edges of interest.Knowledge Discovery Toolbox (KDT), a Python library for parallel graph computations, is customizable in two ways. First, the user can write custom graph algorithms by specifying operations between edges and vertices. These programmer-specified operations are called semiring operations due to KDT's underlying linear-algebraic abstractions. Second, the user can customize existing graph algorithms by writing filters that return true for those vertices and edges the user wants to retain during algorithm execution. For high productivity, both semiring operations and filters are written in a high-level language, resulting in relatively low performance due to the bottleneck of having to call into the Python virtual machine for each vertex and edge.In this work, we use the Selective Embedded JIT Specialization (SEJITS) approach to automatically translate semiring operations and filters defined by programmers into a lower-level efficiency language, bypassing the upcall into Python. We evaluate our approach by comparing it with the high-performance Combinatorial BLAS engine, and show our approach enables users to write in high-level languages and still obtain the high performance of low-level code. We also present a new roofline model for graph traversals, and show that our high-performance implementations do not significantly deviate from the roofline. Overall, we demonstrate the first known solution to the problem of obtaining high performance from a productivity language when applying graph algorithms selectively on semantic graphs.

Graphulo implementation of server-side sparse matrix multiply in the Accumulo database

“…The Knowledge Discovery Toolkit (KDT) distributedmemory Python graph library offers sparse matrix multiplication in a similar design as Graphulo's [19]. Both support custom addition, multiplication and filter operators written in a high level language.…”

Parallel processing of filtered queries in attributed semantic graphs

“…Section 10 gives our conclusions and some remarks on future directions and problems. This paper expands on work first published as a conference paper at IPDPS [11]. 4…”

A Work-Efficient Parallel Sparse Matrix-Sparse Vector Multiplication Algorithm

“…This "frontier expansion" pattern is neatly captured by the SpMSpV primitive: the current frontier is represented with the input vector x, the graph is represented by the matrix A and the next frontier is represented by y. For this reason, SpMSpV is the workhorse of many graph algorithms that are implemented using matrix primitives, such as breadthfirst search [3], maximal independent sets [4], connected components [5], and bipartite graph matching [6]. This makes SpMSpV one of the most important primitives in the upcoming GraphBLAS [7] standard (http://graphblas.org).…”