Versatile and scalable parallel histogram construction

Abstract

Histograms are used in various fields to quickly profile the distribution of a large amount of data. However, it is challenging to efficiently utilize abundant parallel resources in modern processors for histogram construction. To make matters worse, the most efficient implementation varies depending on input parameters (e.g., input distribution, number of bins, and data type) or architecture parameters (e.g., cache capacity and SIMD width).This paper presents versatile histogram methods that achieve competitive performance across a wide range of input types and target architectures. Our open source implementations are highly optimized for various cases and are scalable for more threads and wider SIMD units. We also show that histogram construction can be significantly accelerated by Intel R Xeon Phi TM coprocessors for common input data sets because of their compute power from many cores and instructions for efficient vectorization, such as gather-scatter.For histograms with 256 fixed-width bins, a dual-socket 8-core Intel R Xeon R E5-2690 achieves 13 billion bin updates per second (GUPS), while a 60-core Intel R Xeon Phi TM 5110P coprocessor achieves 18 GUPS for a skewed input. For histograms with 256 variable-width bins, the Xeon processor achieves 4.7 GUPS, while the Xeon Phi coprocessor achieves 9.7 GUPS for a skewed input. For text histogram, or word count, the Xeon processor achieves 342.4 million words per seconds (MWPS). This is 4.12×, 3.46× faster than phoenix and tbb. The Xeon phi processor achieves 401.4 MWPS, which is 1.17× faster than the Xeon processor. Since histogram construction captures essential characteristics of more general reduction-heavy operations, our approach can be extended to other settings.

Fast segmented sort on GPUs

“…It is crucial to design an ecient binning approach on GPUs. Such an approach usually needs carefully designed histogram and scan kernels, such as those proposed in previous research [25,47]. We adopt a simple and ecient "histogram-scanbin" strategy generating the bins across GPU memory hierarchy.…”

Optimizing Indirect Memory References with milk

“…We show end-to-end speedups, ranging from 1.4× on CC and 2× for BFS, to 2.7× on PR, and analyze Milk's performance on synthesized random graphs with variable working set size and temporal and spatial locality. Figure 10 shows speedups when vertices range from V=2 21 to V=2 25 , with an average degree of 16 in uniform degree distribution (i.e., Uniform 21..25 ). We show two BFS variants -BFSd from Graph500 [21] (in Figure 5), as well as BFSp from GAPBS (Figure 8a).…”

“…In Figure 11, we compare performance while varying vertex degrees to show sufficient spatial locality can be exploited even at a low average degree. We use the Count-Degree (Histogram) kernel of Graph500, shown in Figure 6, which is used to construct a graph data structure from a list of edges in graph applications, and is critical in other applications [25]. Figure 11 shows speedups on Histogram in uniform distributions with average degrees from 1 to 64.…”

Exploiting commutativity to reduce the cost of updates to shared data in cache-coherent systems

“…For instance, Jung et al [32] propose parallel histogram implementations using both atomic operations and privatization. These codes process a set of input values, and produce a histogram with a given number of bins.…”

“…For example, consider a loop that processes a set of input values (e.g., image pixels) and produces a histogram of these values with a given number of bins. In this case, the reduction variable is the whole histogram array, and the reduction phase can dominate execution time [32], as shown in Fig. 2.…”