Optimistic parallelism requires abstractions

Abstract

Irregular applications, which manipulate large, pointer-based data structures like graphs, are difficult to parallelize manually. Automatic tools and techniques such as restructuring compilers and runtime speculative execution have failed to uncover much parallelism in these applications, in spite of a lot of effort by the research community. These difficulties have even led some researchers to wonder if there is any coarse-grain parallelism worth exploiting in irregular applications.In this paper, we describe two real-world irregular applications: a Delaunay mesh refinement application and a graphics application that performs agglomerative clustering. By studying the algorithms and data structures used in these applications, we show that there is substantial coarse-grain, data parallelism in these applications, but that this parallelism is very dependent on the input data and therefore cannot be uncovered by compiler analysis. In principle, optimistic techniques such as thread-level speculation can be used to uncover this parallelism, but we argue that current implementations cannot accomplish this because they do not use the proper abstractions for the data structures in these programs.These insights have informed our design of the Galois system, an object-based optimistic parallelization system for irregular applications. There are three main aspects to Galois: (1) a small number of syntactic constructs for packaging optimistic parallelism as iteration over ordered and unordered sets, (2) assertions about methods in class libraries, and (3) a runtime scheme for detecting and recovering from potentially unsafe accesses to shared memory made by an optimistic computation.We show that Delaunay mesh generation and agglomerative clustering can be parallelized in a straight-forward way using the Galois approach, and we present experimental measurements to show that this approach is practical. These results suggest that Galois is a practical approach to exploiting data parallelism in irregular programs.

STAMP: Stanford Transactional Applications for Multi-Processing

“…A few larger workloads that target TM systems have been designed to address the disparity between microbenchmarks and full applications: Delaunay mesh generation [35], database management [14], BerkeleyDB [12], maze routing [40], and Delaunay mesh refinement and agglomerative clustering [24]. These applications represent realistic workloads and avoid the pitfalls of microbenchmarks.…”

“…The usage of transactions in yada is similar to that in [24], but it is applied to a different algorithm in this benchmark. Accesses to the work queue are enclosed by a transaction as is the entire refinement of a skinny triangle.…”

Enhancing and Evaluating the Configuration Capability of a Skeleton for Irregular Computations

“…The experiments were performed in a system with 2 Intel Xeon E5-2660 Sandy Bridge-EP CPUs (8 cores/CPU) at 2.2 GHz and 64 GB of RAM, using g++ 4.7.2 with optimization flag −O3. The graph, node and edge classes used were taken from the Galois system [3], as they were found to be more efficient than the locally developed ones used in [5] and the skeleton transparently supports any classes. The inputs were a road map of the USA with 24 million nodes and 58 million edges for Boruvka, IS and ST, a road map of New York City with 264 thousand nodes and 733 thousand edges for SSSP -both maps taken from [22]-and a mesh with 1 million triangles taken from the Galois project for DMR.…”

“…RELATED WORK While we have not found any other skeleton-based approach oriented to the parallelization of this kind of applications, there are proposals with this aim. The Galois system [3] is a framework for this kind of algorithms that relies on user annotations that describe the properties of the operations. Its interface can be simplified though, if only cautious and unordered algorithms are considered.…”

“…For example, [3] proposes a framework that relies on user annotations that describe the properties of the operations, while [4] is based on a language to describe the evolution of the working set associated to each parallel task. This paper focuses on [5], a parallel algorithmic skeleton [6] [7] that allows to parallelize a large class of irregular applications with little programmer effort by applying a data parallel approach extended with new abstractions [8].…”