Two-Level Main Memory Co-Design: Multi-threaded Algorithmic Primitives, Analysis, and Simulation

Abstract

A fundamental challenge for supercomputer architecture is that processors cannot be fed data from DRAM as fast as CPUs can consume it. Therefore, many applications are memory-bandwidth bound. As the number of cores per chip increases, and traditional DDR DRAM speeds stagnate, the problem is only getting worse. A variety of non-DDR 3D memory technologies (Wide I/O 2, HBM) offer higher bandwidth and lower power by stacking DRAM chips on the processor or nearby on a silicon interposer. However, such a packaging scheme cannot contain sufficient memory capacity for a node. It seems likely that future systems will require at least two levels of main memory: high-bandwidth, low-power memory near the processor and low-bandwidth high-capacity memory further away. This near memory will probably not have significantly faster latency than the far memory. This, combined with the large size of the near memory (multiple GB) and power constraints, may make it difficult to treat it as a standard cache.In this paper, we explore some of the design space for a user-controlled multi-level main memory. We present algorithms designed for the heterogeneous bandwidth, using streaming to exploit data locality. We consider algorithms for the fundamental application of sorting. Our algorithms asymptotically reduce memory-block transfers under certain architectural parameter settings. We use and extend Sandia National Laboratories' SST simulation capability to demonstrate the relationship between increased bandwidth and improved algorithmic performance. Memory access counts from simulations corroborate predicted performance. This co-design effort suggests implementing two-level main memory systems may improve memory performance in fundamental applications.

Optimizing for KNL Usage Modes When Data Doesn't Fit in MCDRAM

“…• We corroborate the simulation results of [4] with real KNL runs. • We give a muiltilevel memory sorting algorithm that leverages MCDRAM to achieve speedups over the best current algorithms run in hardware cache mode, and show that these gains can be preserved without explicitly copying between DDR and MCDRAM.…”

“…Bender, et al discussed the problem of algorithm design for general multi-level memory comprising a main memory and a high bandwidth 'near memory' [4]. They introduced a theoretical model similar to the DAM model, designed and analyzed a sorting algorithm using that model, and used simulation to predict the performance of a simplified version of that algorithm on KNL before the hardware was available.…”

“…For large core counts, they predicted that sorting can indeed be memory bandwidth-bound, and hence stands to gain from MCDRAM. [4] 2.4 KNL and MCDRAM Performance Analyses MCDRAM and HBM represent only a subset of the MLM on modern machines. The Graphics Processing Units (GPUs) now widely used in high-performance computing usually require a fast "device memory" that supplements the main memory of the host processor.…”

High Performance Computing Co-Design Strategies

“…A key architectural change in KNL is the integration of Micron's Multi-Channel-DRAM, which provides a high bandwidth scratchpad memory albeit of limited capacity. In response to this pending architectural change, a Sandia and University team collaborated on an algorithmic and architectural analysis of how to refactor a sorting algorithm to leverage the capabilities of the KNL's two-level memory system [11]. With DOE support, this type of analysis will expand to cover more applications and algorithms.…”

Sparse Matrix-Matrix Multiplication on Multilevel Memory Architectures: Algorithms and Experiments

“…Bender et al. considered two-level memory and corroborated the need for multilevel algorithms using simulation [5]. That particular study focused on sorting algorithms and was limited to simulation due to the unavailability of hardware.…”