On sorting and load balancing on GPUs

Abstract

In this paper we take a look at GPU-Quicksort, an efficient Quicksort algorithm suitable for the highly parallel multi-core graphics processors. Quicksort had previously been considered an inefficient sorting solution for graphics processors, but GPU-Quicksort often performs better than the fastest known sorting implementations for graphics processors, such as radix and bitonic sort. Quicksort can thus be seen as a viable alternative for sorting large quantities of data on graphics processors. We also take look at a comparison of different load balancing schemes. To get maximum performance on the many-core graphics processors it is important to have an even balance of the workload so that all processing units contribute equally to the task at hand. This can be hard to achieve when the cost of a task is not known beforehand and when new sub-tasks are created dynamically during execution. With the recent advent of scatter operations and atomic hardware primitives it is now possible to bring some of the more elaborate dynamic load balancing schemes from the conventional SMP systems domain to the graphics processor domain.

gProximity: Hierarchical GPU‐based Operations for Collision and Distance Queries

“…Other approaches for work distributions on GPUs have explored lock‐free work queues and work stealing approaches [CT08]. We have found that these methods work well as long as parallel tasks are relatively heavy‐weight, such as the octree construction in [CT08], and our results for BVH construction are that work stealing provides roughly equivalent or slightly faster execution time compared to work balancing. However, for our collision and distance queries each operation is far more fine‐grained and even efficient work queue methods have overhead that dominates the overall computation as shown in Fig.…”

“…LGS * 09]. Other methods have explored lock-free queues and work stealing to parallelize octree construction [CT08] [ZHKM08]. The underlying planner performs distance queries repeatedly to compute such a path.…”

“…Previous techniques for GPU work queues used explicit compaction methods between kernel calls [ZHWG08, LGS*09]. Other methods have explored lock‐free queues and work stealing to parallelize octree construction [CT08] and found work stealing to perform best. Overall, the overhead of these methods makes them efficient only for applications with relatively high computational intensity and coarse‐grained parallelism such as hierarchy construction.…”

Approaches for parallelizing reductions on modern GPUs

“…Many application studies have evaluated the use of atomic operations [31,22,6]. Focusing on the parallel Push-Relabel algorithm, Vineet et al [31] used atomic operations to address the data consistency problem.…”

StreamScan

“…Blelloch [1] first introduced scan as a fundamental primitive and discussed its possible applications [2].Later on, more and more parallel applications of scan emerged, such as sort [6,7,8,9,10,11], BFS [12,13], SpMV [13,14], parallel compaction [15],minimal spanning tree [16] and linked list prefix computations [17], etc. The (inclusive) scan problem is defined as follows: Given a sequence with n input elements: ] , ,..., , [ The ⊕ symbol denotes a binary reduction operator, which satisfies the associative law and commutative law.…”