Design and Evaluation of Scalable Concurrent Queues for Many-Core Architectures

Abstract

As core counts increase and as heterogeneity becomes more common in parallel computing, we face the prospect of programming hundreds or even thousands of concurrent threads in a single shared-memory system. At these scales, even highly-efficient concurrent algorithms and data structures can become bottlenecks, unless they are designed from the ground up with throughput as their primary goal.In this paper, we present three contributions: (1) a characterization of queue designs in terms of modern multi-and many-core architectures, (2) the design of a high-throughput, linearizable, blocking, concurrent FIFO queue for many-core architectures that avoids the bottlenecks and pitfalls common in modern queue designs, and (3) a thorough evaluation of concurrent queue throughput across CPU, GPU, and co-processor devices. Our evaluation shows that focusing on throughput, rather than progress guarantees, allows our queue to scale to as much as three orders of magnitude (1000×) faster than lock-free and combining queues on GPU platforms and two times (2×) faster on CPU devices. These results deliver critical insights into the design of data structures for highly concurrent systems: (1) progress guarantees do not guarantee scalability, and (2) allowing an algorithm to block can increase throughput.

A Synchronization-Free Algorithm for Parallel Sparse Triangular Solves

“…The major reason is that the Maxwell architecture dramatically improves its micro-architectures for faster atomic operations, which are extensively utilized in our approach. Actually, Scogland and Feng [25] also confirmed that atomic operations have been continuously improved in the last generations of modern GPUs. Moreover, although the AMD Fury X GPU has higher bandwidth than the NVIDIA Titan X, it is in general slower for our synchronizationfree SpTRSV algorithm.…”

Fast synchronization‐free algorithms for parallel sparse triangular solves with multiple right‐hand sides

“…The major reason is that the Pascal architecture is equipped with higher bandwidth and improved micro-architectures for atomic operations, which are extensively utilized in our approach. Actually, Scogland and Feng [39] also confirmed that atomic operations have been continuously improved in the latest generations of modern GPUs. Moreover, although the AMD Fury X GPU has slightly higher bandwidth than the NVIDIA Titan X, it is in general slower for our synchronization-free SpTRSV algorithm.…”

Data-Parallel Hashing Techniques for GPU Architectures

“…An extensive body of work has embarked on the redesign of data structures for construction and general computation on the GPU [88]. Within the context of searching, these acceleration structures include sorted arrays [3], [4], [8], [51], [66], [67], [98] and linked lists [116], hash tables (see section III), spatial-partitioning trees (e.g., k-d trees [57], [115], [120], octrees [57], [119], bounding volume hierarchies (BVH) [57], [64], R-trees [71], and binary indexing trees [59], [99]), spatial-partitioning grids (e.g., uniform [36], [53], [62] and two-level [52]), skiplists [81], and queues (e.g., binary heap priority [43] and FIFO [17], [101]). Due to significant architectural differences between the CPU and GPU, search structures cannot simply be "ported" from the CPU to the GPU and maintain optimal performance.…”