Accelerating sparse matrix-matrix multiplication with 3D-stacked logic-in-memory hardware

Abstract

Abstract-This paper introduces a 3D-stacked logic-in-memory (LiM) system to accelerate the processing of sparse matrix data that is held in a 3D DRAM system. We build a customized content addressable memory (CAM) hardware structure to exploit the inherent sparse data patterns and model the LiM based hardware accelerator layers that are stacked in between DRAM dies for the efficient sparse matrix operations. Through silicon vias (TSVs) are used to provide the required high inter-layer bandwidth. Furthermore, we adapt the algorithm and data structure to fully leverage the underlying hardware capabilities, and develop the necessary design framework to facilitate the design space evaluation and LiM hardware synthesis. Our simulation demonstrates more than two orders of magnitude of performance and energy efficiency improvements compared with the traditional multithreaded software implementation on modern processors.

A synthesis methodology for application-specific logic-in-memory designs

“…A similar operation is performed for assembling C by using a single "vertical CAM," which activates individual horizontal CAM blocks only if their corresponding column indices are matched. High-level system simulations in [12] show that such a LiM based CAM-SpGEMM core can be used as a low-power hardware accelerator in 3D IC stacks. Sparse matrices are decomposed into sub-blocks and then mapped to DRAM rows for maximizing off-chip DRAM row buffer hit.…”

“…To improve the column-by-column algorithm for SpGEMM, Zhu et al explored the data storage and access patterns in [12] and showed that the SpGEMM operations can be effectively mapped to LiM based content addressable memory (CAM) blocks. As matrix sparsity requires storing only the non-zero elements that are accompanied by their row and column indices, the single cycle "matching" capability of CAMs facilitates index comparison and alignment.…”

“…There are various examples of "processor-in-memory" designs wherein processing units are placed in memory abstraction [2] [3], or more recently, near data computing in 3D and 2.5D stacks to provide more bandwidth with less energy [11]. It has also been shown that various data intensive applications highly benefit from LiM blocks [12] [13]. However, there remains the need for physical synthesis (rather than compilation) of LiM blocks that are compatible with a full chip physical synthesis and offer the ability of system-level exploration.…”

A 3D-stacked logic-in-memory accelerator for application-specific data intensive computing

“…At the bottom of the Fig. 6, we show the structures of LiM core customized for the 2D FFT and SpGEMM respectively [4], [33]. As we can see, both LiM cores involve embedded memory arrays, on-chip buffers, arithmetic units, as well as the control models such as DRAM to Local Memory (D2L) and Local Memory to Core (L2C).…”

“…The CAM based SpGEMM is designed to match the specific sparse data access pattern, and it is able to process the sparse data in an extremely high throughput to match the TSV bandwidth. The design details are beyond the scope of this paper and can be found in another accompanying work [33].…”