Exploiting Oxide Based Resistive RAM Variability for Bayesian Neural Network Hardware Design

Malhotra, Akul; Lu, Sen; Yang, Kezhou; Sengupta, Abhronil

doi:10.1109/tnano.2020.2982819

Cited by 30 publications

(31 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[104] Synaptic sampling, instead, allows the deployment of Bayesian learning and inference, and a number of different hardware platforms, including memristors, have been proposed for this purpose. [105][106][107][108] In the following, we first review the role of probabilistic spiking models for learning, and then provide a short discussion of Bayesian methods. This discussion is aimed at offering some guideline on the development of suitable hardware platforms and on the exploration of properties of memristive devices that typically seen as disadvantageous.…”

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

“…As mentioned, synaptic sampling enables the implementation of Bayesian inference and learning. [ 105–108 ] Unlike the conventional approach considered thus far of identifying a single set of parameter vectors during learning, the Bayesian principle prescribes the inference of a probability distribution over the synaptic weights. While in the presence of sufficient data, this distribution concentrates on the optimal weight configuration, when data are limited, the synaptic weight distribution provides a “credibility” profile in the parameter space.…”

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

“…Initial efforts toward the implementation of Bayesian methods on hardware include references. [ 105–108 ] These papers implement synaptic sampling by including additional circuitry. In contrast, we envision that the inherent randomness of switching processes in memristive devices could provide a source of randomness “for free.” Memristors may, hence, be the missing piece that will unlock the potential of spike‐based computing.…”

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

See 3 more Smart Citations

Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing

Mehonić¹,

Sebastian

Rajendran

et al. 2020

Advanced Intelligent Systems

183

121

View full text Add to dashboard Cite

Three factors are currently driving the main developments in artificial intelligence (AI): availability of vast amounts of data, continuous growth in computing power, and algorithmic innovations. Graphics processing units (GPUs) have been demonstrated as effective co-processors for the implementation of machine learning (ML) algorithms based on deep learning (DL). Solutions based on DL and GPU implementations have led to massive improvements in many AI tasks, but have also caused an exponential increase in demand for computing power. Recent analyses show that the demand for computing power has increased by a factor of 300 000 since 2012, and the estimate is that this demand will double every 3.4 months-at a much faster rate than improvements made historically through Moore's scaling (a sevenfold improvement over the same period of time). [1] At the same time, Moore's law has been slowing down significantly for the last few years, [2] as there are strong indications that we will not be able to continue scaling down complementary metal-oxide-semiconductor (CMOS) transistors. This calls for the exploration of alternative technology roadmaps for the development of scalable and efficient AI solutions. Transistor scaling is not the only way to improve computing performance. Architectural innovations, such as GPUs, field-programmable arrays (FPGAs), and application-specific integrated circuits (ASICs), have all significantly advanced the ML field. [3] A common aspect of modern computing architectures for ML is a move away from the classical von-Neumann architecture that physically separates memory and computing. This approach yields a performance bottleneck that is often the main reason for both energy and speed inefficiency of ML implementations on conventional hardware platforms due to

show abstract

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

Section: Harnessing Hardware Randomness For Learningmentioning

confidence: 99%

See 2 more Smart Citations

Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing

Mehonić¹,

Sebastian

Rajendran

et al. 2020

Advanced Intelligent Systems

183

121

View full text Add to dashboard Cite

show abstract

“…Bayesian deep networks define the synaptic weights with a sample drawn from a probability distribution (in most cases, Gaussian distributions) with learnt mean and variance and inference based on the sampled weights. In Malhotra et al (2020), the FIGURE 11 | Stochastic random number generation utilizing MTJs with programmable probability. (A) MTJ with VCMA and STT (applied via the voltage V VCMA ).…”

Section: Bayesian Neural Networkmentioning

confidence: 99%

Brain-Inspired Hardware Solutions for Inference in Bayesian Networks

Bagheriye

Kwisthout

2021

Front. Neurosci.

View full text Add to dashboard Cite

The implementation of inference (i.e., computing posterior probabilities) in Bayesian networks using a conventional computing paradigm turns out to be inefficient in terms of energy, time, and space, due to the substantial resources required by floating-point operations. A departure from conventional computing systems to make use of the high parallelism of Bayesian inference has attracted recent attention, particularly in the hardware implementation of Bayesian networks. These efforts lead to several implementations ranging from digital circuits, mixed-signal circuits, to analog circuits by leveraging new emerging nonvolatile devices. Several stochastic computing architectures using Bayesian stochastic variables have been proposed, from FPGA-like architectures to brain-inspired architectures such as crossbar arrays. This comprehensive review paper discusses different hardware implementations of Bayesian networks considering different devices, circuits, and architectures, as well as a more futuristic overview to solve existing hardware implementation problems.

show abstract

Complex Oxides for Brain‐Inspired Computing: A Review

et al. 2022

Self Cite

View full text Add to dashboard Cite

The fields of brain‐inspired computing, robotics, and, more broadly, artificial intelligence (AI) seek to implement knowledge gleaned from the natural world into human‐designed electronics and machines. In this review, the opportunities presented by complex oxides, a class of electronic ceramic materials whose properties can be elegantly tuned by doping, electron interactions, and a variety of external stimuli near room temperature, are discussed. The review begins with a discussion of natural intelligence at the elementary level in the nervous system, followed by collective intelligence and learning at the animal colony level mediated by social interactions. An important aspect highlighted is the vast spatial and temporal scales involved in learning and memory. The focus then turns to collective phenomena, such as metal‐to‐insulator transitions (MITs), ferroelectricity, and related examples, to highlight recent demonstrations of artificial neurons, synapses, and circuits and their learning. First‐principles theoretical treatments of the electronic structure, and in situ synchrotron spectroscopy of operating devices are then discussed. The implementation of the experimental characteristics into neural networks and algorithm design is then revewed. Finally, outstanding materials challenges that require a microscopic understanding of the physical mechanisms, which will be essential for advancing the frontiers of neuromorphic computing, are highlighted.

show abstract

Exploiting Oxide Based Resistive RAM Variability for Bayesian Neural Network Hardware Design

Cited by 30 publications

References 21 publications

Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing

Memristors—From In‐Memory Computing, Deep Learning Acceleration, and Spiking Neural Networks to the Future of Neuromorphic and Bio‐Inspired Computing

Brain-Inspired Hardware Solutions for Inference in Bayesian Networks

Complex Oxides for Brain‐Inspired Computing: A Review

Contact Info

Product

Resources

About