HEIF: Highly Efficient Stochastic Computing-Based Inference Framework for Deep Neural Networks

Li, Zhe; Li, Ji; Ren, Ao; Cai, Ruizhe; Ding, Caiwen; Qian, Xuehai; Draper, J.; Yuan, Bo; Tang, Jian; Qiu, Qinru; Wang, Yanzhi

doi:10.1109/tcad.2018.2852752

Cited by 66 publications

(31 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two bestperforming examples were extracted for SC-DCNN [61], and HEIF is reported in Ref. [62]. The results for our work are presented at 1 V. The black curve indicates energy/accuracy tradeoff in our system when the maximum isolator length δ (and, consequently, the warm-up time for the isolator buffers to fill) is varied, holding constant the amount of data collected on the network output.…”

Section: Evaluation and Resultsmentioning

confidence: 99%

“…The origins of stochastic computing lie in the observation that time series data of stochastic spike trains in the brain could be modeled by stochastic jumps from ground to V dd in a logic circuit [58][59][60]. It is no surprise, then, that neural network structures have been implemented successfully and energy efficiently in recent stochastic computing work [61][62][63][64]. Rather than carrying out high level arithmetic and logic operations to "theoretically predict" a neural network's output, stochastic computing implements neuromorphic models of the network in CMOS circuitry.…”

Section: Application To Neural Networkmentioning

confidence: 99%

“…The first stochastic computing implementation of a deep convolutional neural network, a particularly important neural network topology with broad applications to image processing, was proposed by Ren et alia [61] and further optimized in Refs. [62,65]. These works draw heavily on new ideas in the stochastic computing literature, including massively parallel generation of pseudorandom bitstreams [66], state-machine based nonlinear activation functions [63,67], and aggressive use of correlation insensitivity [68].…”

Section: Application To Neural Networkmentioning

confidence: 99%

“…An alternative is bipolar encoding, where a wire turned on with probability p is said to encode the value 2p − 1 [86]. This approach is used by many others in the field [61,62], but unfortunately it is not clear to us that the OR gate can still be used as a useful nonlinear activation function in this encoding scheme.…”

Section: Dual Architecturementioning

confidence: 99%

See 3 more Smart Citations

Energy-Efficient Stochastic Computing with Superparamagnetic Tunnel Junctions

et al. 2020

View full text Add to dashboard Cite

Superparamagnetic tunnel junctions have emerged as a competitive, realistic nanotechnology to support novel forms of stochastic computation in CMOS-compatible platforms. One of their applications is to generate random bitstreams suitable for use in stochastic computing implementations. We describe a method for digitally programmable bitstream generation based on pre-charge sense amplifiers which is more energy efficient than previously explored alternatives. The energy savings offered by this digital generator survive when we use them as the fundamental units of a neural network architecture. To take advantage of the potential savings, we codesign the algorithm with the circuit, rather than directly transcribing a classical neural network into hardware. The flexibility of the neural network mathematics compensates for explicitly energy efficient choices we make at the device level. The result is a convolutional neural network design operating at ≈ 150 nJ per inference with 97 % performance on MNIST-nearly an order of magnitude more energy efficiency than comparable proposals in the recent literature. * matthew.daniels@nist.gov † mark.stiles@nist.govThe low energy, truly random behavior, ease of control, and established compatibility with complementarymetal-oxide-semiconductor (CMOS) circuitry has led to the use of SMTJs as the basis for a number of novel computing schemes [11][12][13]. SMTJs were proposed to implement the concept of probabilistic bits, or "p-bits," which were leveraged for applications as Bayesian neural networks [14][15][16], invertible Boolean logic [17,18], reservoir computing [19], and Ising network models applied to optimization problems [20,21]. SMTJs were also proposed as stochastic neural units [22] that can interact with synaptic units, emulated by crossbar arrays of mag-

show abstract

Section: Evaluation and Resultsmentioning

confidence: 99%

Section: Application To Neural Networkmentioning

confidence: 99%

Section: Application To Neural Networkmentioning

confidence: 99%

Section: Dual Architecturementioning

confidence: 99%

See 2 more Smart Citations

Energy-Efficient Stochastic Computing with Superparamagnetic Tunnel Junctions

et al. 2020

View full text Add to dashboard Cite

show abstract

“…ReLU function is trendy due to its fast computation and solves diminishing return in backward propagation learning during the CNN training stage. However, no SC equivalent circuit existed for that particular function; thus, Li et al (2018a) proposed a novel SC-based ReLU function block as depicted in Fig. 11A.…”

Section: Sc Relu and Sigmoid Activation Layermentioning

confidence: 99%

Figure 14: Differential MAC. Major overhauling to the SN MAC with counter and differential weight control indexing to pipeline the SC MAC computation.

View full text Add to dashboard Cite

Progress and Application Perspectives of Voltage‐Controlled Magnetic Tunnel Junctions

Shao,

Khalili Amiri

2023

Adv Materials Technologies

View full text Add to dashboard Cite

This article discusses the current state of development, open research opportunities, and application perspectives of electric‐field‐controlled magnetic tunnel junctions that use the voltage‐controlled magnetic anisotropy effect to control their magnetization. The integration of embedded magnetic random‐access memory (MRAM) into mainstream semiconductor foundry manufacturing opens new possibilities for the development of energy‐efficient, high‐performance, and intelligent computing systems. The current generation of MRAM, which uses the current‐controlled spin‐transfer torque (STT) effect to write information, has gained traction due to its nonvolatile data retention and lower integration cost compared to embedded Flash. However, scaling MRAM to high bit densities will likely require a transition from current‐controlled to voltage‐controlled operation. In this perspective, an overview of voltage‐controlled magnetic anisotropy (VCMA) as a promising beyond‐STT write mechanism for MRAM devices is provided and recent advancements in developing VCMA‐MRAM devices with perpendicular magnetization are highlighted. Starting from the fundamental mechanisms, the key remaining challenges of VCMA‐MRAM, such as increasing the VCMA coefficient, controlling the write error rate, and achieving field‐free VCMA switching are discussed. Then potential solutions are discussed and open research questions are highlighted. Lastly, prospective applications of voltage‐controlled magnetic tunnel junctions (VC‐MTJs) in security applications, extending beyond their traditional role as memory devices are explored.

show abstract

HEIF: Highly Efficient Stochastic Computing-Based Inference Framework for Deep Neural Networks

Cited by 66 publications

References 40 publications

Energy-Efficient Stochastic Computing with Superparamagnetic Tunnel Junctions

Energy-Efficient Stochastic Computing with Superparamagnetic Tunnel Junctions

Figure 14: Differential MAC. Major overhauling to the SN MAC with counter and differential weight control indexing to pipeline the SC MAC computation.

Progress and Application Perspectives of Voltage‐Controlled Magnetic Tunnel Junctions

Contact Info

Product

Resources

About