A Programmable Heterogeneous Microprocessor Based on Bit-Scalable In-Memory Computing

Abstract

In-memory computing (IMC) addresses the cost of accessing data from memory in a manner that introduces a tradeoff between energy/throughput and computation signal-tonoise ratio (SNR). However, low SNR posed a primary restriction to integrating IMC in larger, heterogeneous architectures required for practical workloads due to the challenges with creating robust abstractions necessary for the hardware and software stack. This work exploits recent progress in high-SNR IMC to achieve a programmable heterogeneous microprocessor architecture implemented in 65-nm CMOS and corresponding interfaces to the software that enables mapping of application workloads. The architecture consists of a 590-Kb IMC accelerator, configurable digital near-memory-computing (NMC) accelerator, RISC-V CPU, and other peripherals. To enable programmability, microarchitectural design of the IMC accelerator provides the integration in the standard processor memory space, areaand energy-efficient analog-to-digital conversion for interfacing to NMC, bit-scalable computation (1-8 b), and input-vector sparsity-proportional energy consumption. The IMC accelerator demonstrates excellent matching between computed outputs and idealized software-modeled outputs, at 1b TOPS/W of 192|400 and 1b-TOPS/mm 2 of 0.60|0.24 for MAC hardware, at V D D of 1.2|0.85 V, both of which scale directly with the bit precision of the input vector and matrix elements. Software libraries developed for application mapping are used to demonstrate CIFAR-10 image classification with a ten-layer CNN, achieving accuracy, throughput, and energy of 89.3%|92.4%, 176|23 images/s, and 5.31|105.2 µJ/image, for 1|4 b quantization levels. Index Terms-Charge-domain compute, deep learning, hardware accelerators, in-memory computing (IMC), neural networks (NNs). I. INTRODUCTION M ACHINE-LEARNING inference, particularly based on neural networks (NNs), has provided unprecedented capabilities in various cognitive tasks, such as vision and language processing. [1]-[5]. However, pervasive deployment, especially in edge applications, has been limited by the high computational requirements that are dominated by highdimensionality matrix-vector multiplications (MVMs). To address this, many optimizations, focusing on both hardware specialization and model design (e.g., sparsity, compression,

10T SRAM Computing-in-Memory Macros for Binary and Multibit MAC Operation of DNN Edge Processors

“…The accuracy loss compared to the simulation is less than 0.5%. Recent works show large capacity SRAM-CIM macros, for example, 64 Kb [28], 384 Kb [29], and 4.5 Mb [33], which allows the application to speech recognition and image classification. Works [19] and [23] report the inference results using the CIFAR-10 dataset, which demands a relatively large network with many inputs.…”

“…Recently, approaches based on computing-in-memory (CIM) have been reported to solve the memory wall problem [16]- [33]. The SRAM-based CIM (SRAM-CIM) structure allowing for the MAC operation within the memory provides two main benefits.…”

An SRAM-Based Multibit In-Memory Matrix-Vector Multiplier With a Precision That Scales Linearly in Area, Time, and Power

“…In addition to the pure analog IMC approaches listed in Table III, a very interesting combination of a binary IMC approach with digital techniques to increase precision was presented in [33]. Specifically, through the use of digital shift and add circuitry, binary-only analog accelerators as, for example, in [31] and [17], can gain linear scalability in terms of weight and input quantizations.…”

A survey of in-spin transfer torque MRAM computing

“…With respect to the circuit-level implementation of storage blocks, memory subsystems can perform the storage function as well as the associated arithmetic and computing units. IMC and NMC were investigated, and the majority of them underwent silicon verification in SRAM [4,[18][19][20][21][22] and several NVMs, including RRAM [23][24][25], STT-MRAM [26][27][28][29][30][31], spin-orbit torque (SOT), and MRAM [32][33][34].…”