Device-aware inference operations in SONOS nonvolatile memory arrays

Bennett, Christopher H.; Xiao, T. Patrick; Dellana, Ryan; Feinberg, Benjamin A.; Agarwal, Sapan; Marinella, Matthew; Agrawal, Vikesh; Prabhakar, V.; Ramkumar, K.; Hinh, Long; Saha, Swatilekha; Raghavan, Vijay R.; Chettuvetty, Ramesh

doi:10.1109/irps45951.2020.9129313

Cited by 13 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our recurrent neural network design, newly developed for this work and visible in The RNN core crossbar is connected to a read-out/logit crossbar of dimensions D o x L, where L is the number of classes (here L = 10); critically, the second core is only activated at the final time step (every t steps). In our resilience testing strategy, we have considered two classes of pre-trained networks: noise-prepared or regularized networks, which train with jittered gaussian filters on every hidden neuron's ReLu function during training following the scheme given in [14] and standard/un-prepared networks which have been trained without noise. As first suggested in [15], the use of noise regularization provides a definitive improvement in inference (Test ) performance of models to internal and external noise or perturbation effects.…”

Section: Methodsmentioning

confidence: 99%

Evaluating complexity and resilience trade-offs in emerging memory inference machines

Bennett,

Dellana,

Xiao

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Neuromorphic-style inference only works well if limited hardware resources are maximized properly, e.g. accuracy continues to scale with parameters and complexity in the face of potential disturbance. In this work, we use realistic crossbar simulations to highlight that compact implementations of deep neural networks are unexpectedly susceptibilie to collapse from multiple system disturbances. Our work proposes a middle path towards high performance and strong resilience utilizing the Mosaics framework, and specifically by re-using synaptic connections in a recurrent neural network implementation that possesses a natural form of noise-immunity.

show abstract

Section: Methodsmentioning

confidence: 99%

Evaluating complexity and resilience trade-offs in emerging memory inference machines

Bennett,

Dellana,

Xiao

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Analog hardware represents digital values in analog quantities like voltages or light pulses and performs computation in analog domain [5]. This form of computation is cheap and projects a 2X performance improvement over digital hardware in speed and energy efficiency [13], [14] as they can achieve projected throughput of multiple tera-operations (TOPs) per seconds and femto-joule energy budgets per multiply-and-accumulate (MAC) operation [4], [15]- [17].…”

Section: Introductionmentioning

confidence: 99%

Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models

Fagbohungbe¹,

Qian²

2022

Preprint

View full text Add to dashboard Cite

The fast execution speed and energy efficiency of analog hardware has made them a strong contender for deployment of deep learning model at the edge. However, there are concerns about the presence of analog noise which causes changes to the weight of the models, leading to performance degradation of deep learning model, despite their inherent noise resistant characteristics. The effect of the popular batch normalization layer on the noise resistant ability of deep learning model is investigated in this work. This systematic study has been carried out by first training different models with and without batch normalization layer on CIFAR10 and CIFAR100 dataset. The weights of the resulting models are then injected with analog noise and the performance of the models on the test dataset is obtained and compared. The results show that the presence of batch normalization layer negatively impacts noise resistant property of deep learning model and the impact grows with the increase of the number of batch normalization layers.

show abstract

“…This method of computation is cheap and projects a 2X performance improvement over digital hardware in speed and energy efficiency [13], [14]. These improvements are because they can achieve projected throughput of multiple tera-operations (TOPs) per seconds and femto-joule energy budgets per multiply-and-accumulate (MAC) operation [4], [15]- [17].…”

Section: Introductionmentioning

confidence: 99%

The Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models

Fagbohungbe

Qian

2022

IEEE Access

View full text Add to dashboard Cite

Office of the Under Secretary of Defense for Research and Engineering (OUSD(R&E)) under agreement number FA8750-15-2-0119. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Office of the Under Secretary of Defense for Research and Engineering (OUSD(R&E)) or the U.S. Government.

show abstract

Device-aware inference operations in SONOS nonvolatile memory arrays

Cited by 13 publications

References 21 publications

Evaluating complexity and resilience trade-offs in emerging memory inference machines

Evaluating complexity and resilience trade-offs in emerging memory inference machines

Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models

The Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models

Contact Info

Product

Resources

About