Gearbox

Abstract

Processing-in-memory (PIM) minimizes data movement overheads by placing processing units near each memory segment. Recent PIMs employ processing units with a SIMD architecture. However, kernels with random accesses, such as sparse-matrix-dense-vector (SpMV) and sparse-matrix-sparse-vector (SpMSpV), cannot effectively exploit the parallelism of SIMD units because SIMD's ALUs remain idle until all the operands are collected from local memory segments (memory segment attached to the processing unit) or remote memory segments (other segments of the memory).For SpMV and SpMSpV, properly partitioning the matrix and the vector among the memory segments is also very important. Partitioning determines (i) how much processing load will be assigned to each processing unit and (ii) how much communication is required among the processing units.In this paper, first, we propose a highly parallel architecture that can exploit the available parallelism even in the presence of random accesses. Second, we observed that, in SpMV and SpMSpV, most of the remote accesses become remote accumulations with the right choice of algorithm and partitioning. The remote accumulations could be offloaded to be performed by processing units next to the destination memory segments, eliminating idle time due to remote accesses. Accordingly, we introduce a dispatching mechanism for remote accumulation offloading. Third, we propose Hybrid partitioning and associated hardware support. Our partitioning technique enables (i) replacing remote read accesses with broadcasting (for only a small portion of data that will be read by all processing units), (ii) reducing the number of remote accumulations, and (iii) balancing the load.Our proposed method, Gearbox, with just one memory stack, delivers on average (up to) 15.73× (52×) speedup over a server-class GPU, NVIDIA P100, with three stacks of HBM2 memory.

Genome Sequence of Brevibacillus brevis HK544, an Antimicrobial Bacterium Isolated from Soil in Daejeon, South Korea

“…While Rowclone [40] proposes bulk data copy of a row data across different banks, significant data movement induced in data analytics are flexible that Rowclone cannot be effectively utilized. TransPIM [51] and GearBox [31] propose specific network-on-chip (NoC) for efficient DRAM internal data movement for target applications. However, their NoCs consume a large area overhead considering the DRAM area constraint.…”

“…As a result, inevitable memory bottleneck problem drives both industry and academia to reassess the DRAM-based near-memory-processing (NMP) [7], [12], [19], [25], [50] and processing-in-memory (PIM) [13], [14], [16], [21], [22], [30], [31], [32], [34], [38], [48], [49] architectures that increase the internal bandwidth by integrating computational logic and DRAM device/cells closely. NMP architectures integrate homogeneous processing unit (PU) per vault in the base logic die of hybrid memory cube (HMC), supporting flexible dataflow for query operations.…”