Analog architectures for neural network acceleration based on non-volatile memory

Xiao, T. Patrick; Bennett, Christopher H.; Feinberg, Benjamin A.; Agarwal, Sapan; Marinella, Matthew

doi:10.1063/1.5143815

Cited by 128 publications

(81 citation statements)

References 143 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While prior demonstrations rely on additional mathematical manipulations of the generated GRNs to establish control over their 𝜇𝜇 and 𝜎𝜎, we are able to achieve it without any additional manipulations or circuitry [22][23][24]35]. It is common practice in neural network accelerators to use two devices per synapse in order to map both positive and negative weights [37]. Here, the input to the synapse, 𝑉𝑉 in is applied as +𝑉𝑉 in and -𝑉𝑉 in to T + and T − , respectively, as shown in Fig.…”

Section: Gaussian Random Number Generator-based Synapsementioning

confidence: 99%

“…The hardware for activation function in neural accelerators is generally realized using standard CMOS-based analog and digital components, and hence these implementations do not utilize the advantages offered by emerging materials [37]. Moreover, hyperbolic tangent (tanh) and sigmoid functions are highly non-linear, significantly complicating their hardware demonstration [38].…”

Section: Neurons With Modified Hyperbolic Tangent Activation Functionmentioning

confidence: 99%

See 1 more Smart Citation

A Neural Network Accelerator to Avoid Inference Inaccuracy

Sebastian

Das

2022

Preprint

View full text Add to dashboard Cite

Artificial neural networks (ANNs) have demonstrated their superiority over traditional computing architectures in tasks such as pattern classification and learning. While ANNs demonstrate high prediction accuracy, they do not measure uncertainty in predictions, and hence they can make wrong predictions with high confidence, which can be detrimental for many mission-critical applications. In contrast, Bayesian neural networks (BNNs) naturally include such uncertainty in their model. Unlike ANNs, where the synaptic weights are point estimates (single-valued), in BNNs, the weights are represented by probability distributions (e.g. Gaussian distribution). This makes the hardware implementation of BNNs challenging since on-chip Gaussian random number generators (GRNGs) based on silicon complementary metal oxide semiconductor (CMOS) technology are area and energy inefficient. Stochastic switching in memristors can be used to build probabilistic synapses but with very limited tunability. Additionally, memristor technology rely heavily on CMOS-based peripherals to emulate neurons. Here we introduce three-terminal memtransistors based on two-dimensional (2D) materials, which can emulate both probabilistic synapses as well as reconfigurable neurons. The cycle-to-cycle variation in the programming of the 2D memtransistor is exploited to achieve GRNG-based synapses, whereas 2D memtransistor based integrated circuits are used to obtain neurons with hyperbolic tangent and sigmoid activation functions. Finally, memtransistor-based synapses and neurons are combined in a crossbar array architecture to realize a BNN accelerator and the performance is evaluated using the IRIS data classification task.

show abstract

Section: Gaussian Random Number Generator-based Synapsementioning

confidence: 99%

Section: Neurons With Modified Hyperbolic Tangent Activation Functionmentioning

confidence: 99%

A Neural Network Accelerator to Avoid Inference Inaccuracy

Sebastian

Das

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…When the weights are updated element-wise or row-wise in the crossbar array, the time complexity proportionally increases with an increase in the array size. Crossbar-compatible and fully parallel update schemes have thus been proposed to accelerate neural network training ( Burr et al, 2015 ; Gao et al, 2015 ; Kadetotad et al, 2015 ; Gokmen and Vlasov, 2016 ; Xiao et al, 2020 ). For the target crossbar array, by applying update pulses simultaneously to all rows and columns based on the neuron’s local knowledge of X and δ, respectively, the parallel updates in each cross point can be executed by the number of pulse overlaps.…”

Section: Hardware Neural Networkmentioning

confidence: 99%

Neural Network Training Acceleration With RRAM-Based Hybrid Synapses

et al. 2021

View full text Add to dashboard Cite

Hardware neural network (HNN) based on analog synapse array excels in accelerating parallel computations. To implement an energy-efficient HNN with high accuracy, high-precision synaptic devices and fully-parallel array operations are essential. However, existing resistive memory (RRAM) devices can represent only a finite number of conductance states. Recently, there have been attempts to compensate device nonidealities using multiple devices per weight. While there is a benefit, it is difficult to apply the existing parallel updating scheme to the synaptic units, which significantly increases updating process’s cost in terms of computation speed, energy, and complexity. Here, we propose an RRAM-based hybrid synaptic unit consisting of a “big” synapse and a “small” synapse, and a related training method. Unlike previous attempts, array-wise fully-parallel learning is possible with our proposed architecture with a simple array selection logic. To experimentally verify the hybrid synapse, we exploit Mo/TiOx RRAM, which shows promising synaptic properties and areal dependency of conductance precision. By realizing the intrinsic gain via proportionally scaled device area, we show that the big and small synapse can be implemented at the device-level without modifications to the operational scheme. Through neural network simulations, we confirm that RRAM-based hybrid synapse with the proposed learning method achieves maximum accuracy of 97 %, comparable to floating-point implementation (97.92%) of the software even with only 50 conductance states in each device. Our results promise training efficiency and inference accuracy by using existing RRAM devices.

show abstract

“…Due to the energy-efficient features of memristors, attempts utilizing memristors to build synapses ( Wang et al, 2017 ) and neurons ( Wang et al, 2018 ) have been made, and achieved great progress. The conductance of RRAM can be modulated by electrical pulses either through a variably conductive filament or through the migration of oxygen vacancies ( Milo, 2020 ; Xiao et al, 2020 ). In addition, RRAM has attractive features, such as high scalability, low consumption power, fast write/read speed, stable storage, and multi-value tune ability.…”

Section: Introductionmentioning

confidence: 99%

MSPAN: A Memristive Spike-Based Computing Engine With Adaptive Neuron for Edge Arrhythmia Detection

et al. 2021

View full text Add to dashboard Cite

In this work, a memristive spike-based computing in memory (CIM) system with adaptive neuron (MSPAN) is proposed to realize energy-efficient remote arrhythmia detection with high accuracy in edge devices by software and hardware co-design. A multi-layer deep integrative spiking neural network (DiSNN) is first designed with an accuracy of 93.6% in 4-class ECG classification tasks. Then a memristor-based CIM architecture and the corresponding mapping method are proposed to deploy the DiSNN. By evaluation, the overall system achieves an accuracy of over 92.25% on the MIT-BIH dataset while the area is 3.438 mm2 and the power consumption is 0.178 μJ per heartbeat at a clock frequency of 500 MHz. These results reveal that the proposed MSPAN system is promising for arrhythmia detection in edge devices.

show abstract

Analog architectures for neural network acceleration based on non-volatile memory

Cited by 128 publications

References 143 publications

A Neural Network Accelerator to Avoid Inference Inaccuracy

A Neural Network Accelerator to Avoid Inference Inaccuracy

Neural Network Training Acceleration With RRAM-Based Hybrid Synapses

MSPAN: A Memristive Spike-Based Computing Engine With Adaptive Neuron for Edge Arrhythmia Detection

Contact Info

Product

Resources

About