A Modern Primer on Processing in Memory

Abstract

Modern computing systems are overwhelmingly designed to move data to computation. This design choice goes directly against at least three key trends in computing that cause performance, scalability and energy bottlenecks:(1) data access is a key bottleneck as many important applications are increasingly data-intensive, and memory bandwidth and energy do not scale well, (2) energy consumption is a key limiter in almost all computing platforms, especially server and mobile systems, (3) data movement, especially off-chip to on-chip, is very expensive in terms of bandwidth, energy and latency, much more so than computation. These trends are especially severely-felt in the data-intensive server and energy-constrained mobile systems of today.At the same time, conventional memory technology is facing many technology scaling challenges in terms of reliability, energy, and performance. As a result, memory system architects are open to organizing memory in different ways and making it more intelligent, at the expense of higher cost. The emergence of 3D-stacked memory plus logic, the adoption of error correcting codes inside the latest DRAM chips, proliferation of different main memory standards and chips, specialized for different purposes (e.g., graphics, low-power, high bandwidth, low latency), and the necessity of designing new solutions to serious reliability and security issues, such as the RowHammer phenomenon, are an evidence of this trend.This chapter discusses recent research that aims to practically enable computation close to data, an approach we call processing-in-memory (PIM). PIM places computation mechanisms in or near where the data is stored (i.e., inside the memory chips, in the logic layer of 3D-stacked memory, or in the memory controllers), so that data movement between the computation units and memory is reduced or eliminated. While the general idea of PIM is not new, we discuss motivating trends in applications as well as memory circuits/technology that greatly exacerbate the need for enabling it in modern computing systems. We examine at least two promising new approaches to designing PIM systems to accelerate important data-intensive applications: (1) processing using memory by exploiting analog operational properties of DRAM chips to perform massively-parallel operations in memory, with low-cost changes, (2) processing near memory by exploiting 3D-stacked memory technology design to provide high memory bandwidth and low memory latency to in-memory logic. In both approaches, we describe and tackle relevant cross-layer research, design, and adoption challenges in devices, architecture, systems, and programming models. Our focus is on the development of in-memory processing designs that can be adopted in real computing platforms at low cost. We conclude by discussing work on solving key challenges to the practical adoption of PIM.

QUAC-TRNG: High-Throughput True Random Number Generation Using Quadruple Row Activation in Commodity DRAM Chips

“…Synergy With PIM. Processing-in-memory (PIM) systems improve system performance and/or energy consumption by performing computations directly within a memory chip, thereby avoiding unnecessary data movement [25,26,57,58,60,116,118,137,139]. Prior works propose a broad range of PIM systems [5-8, 13, 22-24, 34, 38, 44, 48, 49, 54, 55, 58, 59, 65, 66, 71, 72, 89, 98, 100, 103, 107, 113, 115, 119, 120, 124, 133-135, 137-139, 142, 148, 164, 168] in the context of various workloads and memory devices.…”

“…Therefore, QUAC-TRNG o ers a new design point that can enable new applications that were previously infeasible with alternative TRNGs, especially for systems where the costs of on-chip TRNGs may be prohibitive (e.g., heavily constrained embedded systems, processing-in-memory architectures). For example, QUAC-TRNG would enable processing-in-memory systems [62,116,137,157] to execute security workloads as it enables true random number generation directly within a DRAM chip.…”

PIM-Enclave: Bringing Confidential Computation Inside Memory

“…Stacked memory architectures vertically stack DRAM layers on top of each other and connect the vertical partitions of memory using high-bandwidth through-silicon vias (TSVs). A typical 3D-stacked memory configuration can employ thousands of TSVs [45], which makes its internal memory bandwidth far exceed that of traditional memory systems. At the bottom of the memory stacks, there is a logic layer that can host hardware logic that can interact with both the host processor and the DRAM memory.…”

“…Due to its nature of being near memory, applications offloaded to PIM gain a high memory bandwidth as they do not have to move data across the slow memory bus. Moreover, in 3D-stacked memories, TSV connection between the layer naturally provides more internal bandwidth [45]. This makes PIM-based applications exceed memory-bounded workloads and workloads with erratic memory access patterns.…”

SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems

“…This motivates using processing-in-memory (PIM) to gain the much needed speedups in graph mining. While PIM is not the only potential solution for hardware acceleration of graph mining, we select PIM because (1) it represents one of the most promising trends to tackle the memory bottleneck [69,128] outperforming other approaches [153], (2) it offers well-understood designs [129], and (3) numerous works illustrate it brings very large speedups in simple graph algorithms such as BFS or PageRank (see more than 15 works in Table 7), also using processing fully inside DRAM [10]. Yet, graph mining algorithms are much more complex: they employ deep recursion, create many intermediate data structures with non-trivial inter-dependencies, and have high load imbalance [62,186].…”