LLAMA: Efficient graph analytics using Large Multiversioned Arrays

Abstract

Abstract-We present LLAMA, a graph storage and analysis system that supports mutability and out-of-memory execution. LLAMA performs comparably to immutable main-memory analysis systems for graphs that fit in memory and significantly outperforms existing out-of-memory analysis systems for graphs that exceed main memory. LLAMA bases its implementation on the compressed sparse row (CSR) representation, which is a read-only representation commonly used for graph analytics. We augment this representation to support mutability and persistence using a novel implementation of multi-versioned array snapshots, making it ideal for applications that receive a steady stream of new data, but need to perform whole-graph analysis on consistent views of the data. We compare LLAMA to stateof-the-art systems on representative graph analysis workloads, showing that LLAMA scales well both out-of-memory and across parallel cores. Our evaluation shows that LLAMA's mutability introduces modest overheads of 3-18% relative to immutable CSR for in-memory execution and that it outperforms state-ofthe-art out-of-memory systems in most cases, with a best case improvement of 5x on breadth-first-search.

Scaling data mining in massively parallel dataflow systems

“…General data flow systems are able to efficiently run vertex-centric programs, and additionally provide the machinery for conducting relational operations on the graph data. While recent work shows that single-machine solutions easily outperform distributed systems on the dataset sizes used in academia [29,112,119,123], we think that these single machine solutions will stay constrained to academic use cases. In industry scenarios, the value of distributed graph processing on general dataflow systems comes not from the performance of the algorithms, but from the ability to create complex pipelines mixing ETL, machine learning and graph-processing tasks, using a single system.…”

“…Recently, it has been shown that the large networks commonly used in academia can be efficiently processed on single machines [29,112,119,123].…”

An I/O-Efficient Buffer Batch Replacement Policy for Update-Intensive Graph Databases

“…As a result, it introduces unnecessary random I/Os. To deal with this drawback, one recent work has proposed to remove buffer managers [14]. Besides, there are also alternative approaches which utilize index structures such as log structured merge tree [18] or fractal tree [4] to handle update-intensive workload.…”

“…In the figure, the interval [14,14] actually represents an individual data page at the position 14 on disk.…”

WolfPath: Accelerating Iterative Traversing-Based Graph Processing Algorithms on GPU

“…However, since developing distributed graph algorithm is challenging, some researchers divert their attention to design the graph processing system that handle large scale graphs on a single PC. The research endeavours in this direction have delivered the systems such as GraphChi [17], PathGraph [45], GraphQ [39], LLAMA [27] and GridGraph [51]. However, these systems suffer from the limited degree of parallelism in conventional processors.…”