AB9: A neural processor for inference acceleration

Abstract

We present AB9, a neural processor for inference acceleration. AB9 consists of a systolic tensor core (STC) neural network accelerator designed to accelerate artificial intelligence applications by exploiting the data reuse and parallelism characteristics inherent in neural networks while providing fast access to large on‐chip memory. Complementing the hardware is an intuitive and user‐friendly development environment that includes a simulator and an implementation flow that provides a high degree of programmability with a short development time. Along with a 40‐TFLOP STC that includes 32k arithmetic units and over 36 MB of on‐chip SRAM, our baseline implementation of AB9 consists of a 1‐GHz quad‐core setup with other various industry‐standard peripheral intellectual properties. The acceleration performance and power efficiency were evaluated using YOLOv2, and the results show that AB9 has superior performance and power efficiency to that of a general‐purpose graphics processing unit implementation. AB9 has been taped out in the TSMC 28‐nm process with a chip size of 17 × 23 mm2. Delivery is expected later this year.

A layer‐wise frequency scaling for a neural processing unit

“…P dynamic is proportional to α, C, f , and V 2 , as expressed in (2), which correspond to the switching activity, the switching capacitance, the operating frequency, and the square of operating voltage, respectively. Among them, f V 2 is determined by PLLs and on-board PMICs, regardless of operations.…”

“…Our proposed DFS applies the highest allowed under the power limit of an NPU before processing each layer. To determine the highest allowable , we build a power model of an NPU chip (i.e., AB9 [2]) to predict the power consumption of an NPU based on a real measurement; we measure power values for hardware resources such as processing units. Based on the prediction from the power model, our proposed DFS adjusts as high as possible without violating the power limit.…”

“…Recently, neural processing units (NPUs) have emerged to accelerate deep neural networks (DNNs) [1,2]. NPUs exhibit high parallelism in the computation of neural networks (NNs).…”

Live Demonstration: A Neural Processor for AI Acceleration

“…We introduced AB9, an AI processor including a quadcore system-on-chip (SoC) and a systolic tensor core (STC) for the inference acceleration [1]- [3]. The overall system is illustrated in Fig.…”