A Neuromorphic Chip Optimized for Deep Learning and CMOS Technology With Time-Domain Analog and Digital Mixed-Signal Processing

Miyashita, Daisuke; Kousai, Shouhei; Suzuki, Tomoya; Deguchi, Jun

doi:10.1109/jssc.2017.2712626

Cited by 105 publications

(75 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e main di erence with respect to Refs. [19][20][21] is that our approach allows for precise four-quadrant VMM using analog input and weights. Unlike the work presented in Ref.…”

Section: Introductionmentioning

confidence: 99%

Energy-Efficient Time-Domain Vector-by-Matrix Multiplier for Neurocomputing and Beyond

Bavandpour

Mahmoodi

Strukov

2019

IEEE Trans. Circuits Syst. II

View full text Add to dashboard Cite

We propose an extremely energy-e cient mixed-signal approach for performing vector-by-matrix multiplication in a time domain. In such implementation, multi-bit values of the input and output vector elements are represented with time-encoded digital signals, while multi-bit matrix weights are realized with current sources, e.g. transistors biased in subthreshold regime. With our approach, multipliers can be chained together to implement large-scale circuits completely in a time domain. Multiplier operation does not rely on energy-taxing static currents, which are typical for peripheral and input/output conversion circuits of the conventional mixed-signal implementations. As a case study, we have designed a multilayer perceptron, based on two layers of 10 × 10 four-quadrant vectorby-matrix multipliers, in 55-nm process with embedded NOR ash memory technology, which allows for compact implementation of adjustable current sources. Our analysis, based on memory cell measurements, shows that at high computing speed the drain-induced barrier lowering is a major factor limiting multiplier precision to ∼ 6 bit. Post-layout estimates for a conservative 6-bit digital input/output N × N multiplier designed in 55 nm process, including I/O circuitry for converting between digital and time domain representations, show ∼ 7 fJ/Op for N > 200, which can be further lowered well below 1 fJ/Op for more optimal and aggressive design.

show abstract

“…e main di erence with respect to Refs. [19][20][21] is that our approach allows for precise four-quadrant VMM using analog input and weights. Unlike the work presented in Ref.…”

Section: Introductionmentioning

confidence: 99%

Energy-Efficient Time-Domain Vector-by-Matrix Multiplier for Neurocomputing and Beyond

Bavandpour

Mahmoodi

Strukov

2019

IEEE Trans. Circuits Syst. II

View full text Add to dashboard Cite

show abstract

“…This is a major differentiator when compared with the state-of-the-art NN-based AI. Table 4 compares the Tsetlin machine energy efficiency with three recently reported NN approaches: a mixed-signal neuromorphic approach using time-domain arithmetic organized in a spatially unrolled neuron architecture [25], a low-power FPGA-based convolutional BNN (CBNN) approach that uses exclusive NOR (XNOR) adder-based integer weight biases to reduce the arithmetic-heavy batch normalization for synchronization between the deeper layers [26] and finally an in-memory BNN approach using parallel content-addressable memories (CAMs) to reduce the frequent data movement costs [27]. Our comparative analysis considered disparities between these approaches in terms of (i) their internal structures in both combinational and sequential parts and (ii) the size of datasets used to validate the efficiencies.…”

Section: Performance and Energy Efficiencymentioning

confidence: 99%

Learning automata based energy-efficient AI hardware design for IoT applications

Wheeldon

Shafik

Rahman

et al. 2020

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

Energy efficiency continues to be the core design challenge for artificial intelligence (AI) hardware designers. In this paper, we propose a new AI hardware architecture targeting Internet of Things applications. The architecture is founded on the principle of learning automata, defined using propositional logic. The logic-based underpinning enables low-energy footprints as well as high learning accuracy during training and inference, which are crucial requirements for efficient AI with long operating life. We present the first insights into this new architecture in the form of a custom-designed integrated circuit for pervasive applications. Fundamental to this circuit is systematic encoding of binarized input data fed into maximally parallel logic blocks. The allocation of these blocks is optimized through a design exploration and automation flow using field programmable gate array-based fast prototypes and software simulations. The design flow allows for an expedited hyperparameter search for meeting the conflicting requirements of energy frugality and high accuracy. Extensive validations on the hardware implementation of the new architecture using single- and multi-class machine learning datasets show potential for significantly lower energy than the existing AI hardware architectures. In addition, we demonstrate test accuracy and robustness matching the software implementation, outperforming other state-of-the-art machine learning algorithms. This article is part of the theme issue ‘Advanced electromagnetic non-destructive evaluation and smart monitoring’.

show abstract

“…However, the area of the PE unit can be significantly reduced, since the multiplications of a binary input activation and a binary weight can be replaced with bitwise AND operations. Due to the smaller area of PE units and the reduced buffer sizes for storing binary parameters (input activation and weight), the fully parallel architecture [24] is usually adopted. In this architecture, both input activations and weights are preloaded to compute the output activation within one clock.…”

Section: ) Bnn Convolutional Layer Designmentioning

confidence: 99%

“…It consists of the DWM-based cell array for BNN, the accumulators, and the output buffer. In the BNN design, the reduced parameters (input activation and weight) facilitate the spatially unrolled architecture [24], for which the input activations and weights are fully preloaded in the DWM input buses and the DWM-based weight reorder blocks, respectively. Fig.…”

Section: A Dwm-based Bnn Convolutional Layermentioning

confidence: 99%

Domain Wall Memory-Based Design of Deep Neural Network Convolutional Layers

et al. 2020

View full text Add to dashboard Cite

In the hardware implementation of deep learning algorithms such as, convolutional neural networks (CNNs) and binarized neural networks (BNNs), multiple dot products and memories for storing parameters take a significant portion of area and power consumption. In this paper, we propose a domain wall memory (DWM) based design of CNN and BNN convolutional layers. In the proposed design, the resistive cell sensing mechanism is efficiently exploited to design low-cost DWM-based cell arrays for storing parameters. The unique serial access mechanism and small footprint of DWM are also used to reduce the area and energy cost of DWM-based design for filter sliding. Simulation results with 65 nm CMOS process show 45% and 43% of energy savings compared to the conventional CNN and BNN design approach, respectively. INDEX TERMS Binarized neural network, convolutional neural network, deep neural network, domain wall memory.

show abstract

A Neuromorphic Chip Optimized for Deep Learning and CMOS Technology With Time-Domain Analog and Digital Mixed-Signal Processing

Cited by 105 publications

References 13 publications

Energy-Efficient Time-Domain Vector-by-Matrix Multiplier for Neurocomputing and Beyond

Energy-Efficient Time-Domain Vector-by-Matrix Multiplier for Neurocomputing and Beyond

Learning automata based energy-efficient AI hardware design for IoT applications

Domain Wall Memory-Based Design of Deep Neural Network Convolutional Layers

Contact Info

Product

Resources

About