A Multi-Functional In-Memory Inference Processor Using a Standard 6T SRAM Array

Abstract

This paper describes a multi-functional deep in-memory processor for inference applications. Deep inmemory processing is achieved by embedding pitch-matched low-SNR analog processing into a standard 6T 16KB SRAM array in 65 nm CMOS. Four applications are demonstrated. The prototype achieves up to 5.6X (9.7X estimated for multi-bank scenario) energy savings with negligible (≤1%) accuracy degradation in all four applications as compared to the conventional architecture. 2Emerging inference applications require processing of huge data volumes [1]. A conventional inference architecture ( Fig. 1) implements memory access, data transfer from memory to processor, data aggregation, and slicing. In such architectures, memory access energy dominates, e.g., an 8-b SRAM read access and an 8-b MAC consumes 5pJ and 1pJ in 65nm CMOS, respectively. Additionally, the memoryprocessor interface presents a severe throughput bottleneck. Deep in-memory signal processing concept was proposed in [2] to overcome these challenges by embedding mixed-signal processing in the periphery of the SRAM bit-cell array (BCA). However, an IC implementation needs to address a host of new challenges including the stringent row & column pitch-matching requirements imposed by the BCA without altering its storage density or its read/write functionality, and enabling multiple functions with mixed signal circuitry. Recently [3], a single function, 5×1-b in-memory classifier IC has been demonstrated.The proposed deep in-memory inference architecture has four stages ( Fig.1): 1) multi-row functional read (MR-FR), 2) bit-line (BL) processing (BLP), 3) cross BL processing (CBLP), and 4) ADC and slicing. The MR-FR accesses multiple rows in one pre-charge cycle using pulse-width modulated word-line (PWM-WL) signals to generate a BL voltage drop proportional to a weighted sum of multiple bits stored in multiple rows in the column, and also performs word-level add/subtract. The BLP implements reconfigurable column pitch-matched mixed-signal circuits to execute computations such as multiply/absolute value/comparison on the BL voltages, in a massively column-parallel fashion. The CBLP aggregates the BLP outputs into a scalar which is sliced to obtain the final decision. The BLP and CBLP can be reconfigured to operate the architecture in either a dot product (DP) mode or Manhattan distance (MD) mode. Reconfigurable stages enable multiple functions (Fig. 1 table) including normal read/write. The chip architecture (Fig. 2) includes a digital controller (CTRL) and a CORE. The normal Technology Die size CTRL operating freq. SRAM capacity Bitcell dimension Supply voltage SVM 963.1 Matched filter 481.5 KNN 33.6K Template matching 33.6K SVM 1.7M Matched filter 3.4M KNN 54.3K Template matching 54.3K Energy per decision (pJ) Decision Throughput (Decisions/s) 65 nm CMOS 1.2 mm × 1.2 mm 1 GHz 16 KB (1 bank of 512 × 256-b) CORE: 1.0 V, CTRL: 0.85 V 2.11 × 0.92 um 2

8T SRAM Cell as a Multibit Dot-Product Engine for Beyond Von Neumann Computing

“…Memristors based on resistive-RAMs (Re-RAMs) have been reported in many works as an analog dot product compute engine [4], [8]. Few works based on analog computations in SRAM cells can be found in [9], [10]. Both these works use 6T SRAM cells and rely on the resultant accumulated voltage on the bit-lines (BLs).…”

CONV-SRAM: An Energy-Efficient SRAM With In-Memory Dot-Product Computation for Low-Power Convolutional Neural Networks

“…This work ISSCC'17 [9] JSSC'17 [5] JSSC'17 [13] JSSC'18 [14] JSSC'18 [16] CICC'18 [19] ISSCC'18 [20] Tech 1.75 (0.663) 5 11.51 1 (46.0) 5 1.94 6.0 130 (24.12) 5 50.6…”

“…4 Does not include energy to access IFMP/OFMP memories 5 Assuming a 65-nm implementation and Energy ∝ (Tech.) 2 prior work, both conventional digital [5], [9], [19], [20] and in-memory approaches [14], [16]. It should be noted that, while [5], [9], [16], [19], [20] are full systems, the main focus of this work was to demonstrate in-memory computation capability for CNNs.…”

“…The second key feature of our approach is the improvement of the dynamic voltage range for the analog computations without disturbing any bit-cell. In the conventional approach (with 6T SRAM bit-cells) [13], [14], [17], where multiple word-lines (WL) are activated for the same bit-line, there might be a situation where one of the accessed bit-cells in that column is in pseudo-write mode (Fig. 7).…”

“…The third key feature of our work, which distinguishes it from other "in-memory" computing approaches [14], [16], is the use of the inherent bit-line capacitances in the SRAM array to implement the computations. This precludes the need for extra area-intensive capacitors, which would be otherwise required at the SRAM periphery [14] to implement some of the analog computations.…”