14.5 Envision: A 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable Convolutional Neural Network processor in 28nm FDSOI

Abstract

ConvNets, or Convolutional Neural Networks (CNN), are state-of-the-art classification algorithms, achieving near-human performance in visual recognition [1]. New trends such as augmented reality demand always-on visual processing in wearable devices. Yet, advanced ConvNets achieving high recognition rates are too expensive in terms of energy as they require substantial data movement and billions of convolution computations. Today, state-of-the-art mobile GPU's and ConvNet accelerator ASICs [2][3] only demonstrate energy-efficiencies of 10's to several 100's GOPS/W, which is one order of magnitude below requirements for always-on applications. This paper introduces the concept of hierarchical recognition processing, combined with the Envision platform: an energy-scalable ConvNet processor achieving efficiencies up to 10TOPS/W, while maintaining recognition rate and throughput. Envision hereby enables always-on visual recognition in wearable devices. Figure 14.5.1 demonstrates the concept of hierarchical recognition. Here, a hierarchy of increasingly complex individually trained ConvNets, with different topologies, different network sizes and increasing computational precision requirements, is used in the context of person identification. This enables constant scanning for faces at very low average energy cost, yet rapidly scales up to more complex networks detecting a specific face such as a device's owner, all the way up to full VGG-16-based 5760-face recognition. The opportunities afforded by such a hierarchical approach span far beyond face recognition alone, but can only be exploited by digital systems demonstrating wide-range energy scalability across computational precision. State-of-the-art ASICs in references [3] and [4] only show 1.5× and 8.2× energy-efficiency scalability, respectively. Envision improves upon this by introducing subword-parallel Dynamic-Voltage-AccuracyFrequency Scaling (DVAFS), a circuit-level technique enabling 40× energy-precision scalability at constant throughput. Figure 14.5.2 illustrates the basic principle of DVAFS and compares it to Dynamic-Accuracy Scaling (DAS) and Dynamic-Voltage-Accuracy Scaling (DVAS) [4]. In DAS, switching activity and hence energy consumption is reduced for low precision computations by rounding and masking a configurable number of LSB's at the inputs of multiplyaccumulate (MAC) units. DVAS exploits shorter critical paths in DAS's reduced-precision modes by combining it with voltage scaling for increased energy scalability. This paper proposes subword-parallel DVAFS, which further improves upon DVAS, by reusing inactive arithmetic cells at reduced precision. These can be reconfigured to compute 2×1-8b or 4×1-4b (N×1-16b/N, with N the level of subword-parallelism), rather than 1×1-16b words per cycle, when operating at less than 8b precision. At constant data throughput, this permits lowering the processor's frequency and voltage significantly below DVAS values. As a result, DVAFS is a dynamic precision technique which simultaneously lowers all run-time a...

Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices

“…3b), which is often used for a output-stationary dataflow, relies on another [29][30][31][32]: iacts are reused vertically and psums are accumulated horizontally. (b) Temporal accumulation array [15][16][17]: iacts are reused vertically and weights are reused horizontally. set of data dimensions to achieve high compute parallelism.…”

“…While these approaches provide theoretical reductions in the size and number of operations and storage cost, specialized hardware is often necessary to translate these theoretical benefits into measurable improvements in energy efficiency and processing speed. Support for reduced precision has been demonstrated in recent hardware implementations, including Envision [15], Thinker [16], UNPU [17], Loom [18], and Stripes [19]. These works have shown various methods that efficiently translate reduced bitwidth from 16-bits down to 1-bit into both energy savings and increase in processing speed.…”

Bio-mimetic synaptic plasticity and learning in a sub-500 mV Cu/SiO2/W memristor

“…Mimicking human brain's architecture and computational primitives to build intelligent information processing systems is the key goal of neuromorphic engineering research activities worldwide. While there have been several demonstrations of neuromorphic computational platforms using standard complementary metal-oxide semiconductor (CMOS) technology [11][12][13][14][15][16][17][18][19][20] and demonstrations of nanoscale devices to mimic neuronal and synaptic dynamics [21][22][23][24][25][26][27][28][29][30] , none of these have achieved the target energy efficiency specifications necessary to build systems that can learn in real-time and in-the-field 31 .…”

BinarEye: An always-on energy-accuracy-scalable binary CNN processor with all memory on chip in 28nm CMOS

“…Fig. 1 shows embedded GPU's and general Neural Processing Units (NPU) [2,3,4,5] consume too much energy for this task, both in their computations and in their off-chip DRAM access, while classical machine learning methods [6] are either inaccurate or lack the flexibility to cover multiple tasks. This paper introduces BinarEye: a CNN processor optimized for BinaryNets [1]: Neural Networks with weights and activations constrained to +1/-1.…”