Stochastic learning in deep neural networks based on nanoscale PCMO device characteristics

Babu, Anakha V; Lashkare, Sandip; Ganguly, Udayan; Rajendran, Bipin

doi:10.1016/j.neucom.2018.09.019

Cited by 13 publications

(9 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For all combinations of measured SPMs and RPMs, multilayer perceptron simulations are conducted for 6,000 epochs. As described in Section "Stepwise SET and RESET Pulse (Moon et al, 2015;Suri et al, 2015;Ambrogio et al, 2016;Babu et al, 2018;Fumarola et al, 2018;Go et al, 2019;Wu et al, 2020;Yin et al, 2020). (D) Learning curves for PCMO devices with Al and Ta 2 O 5 interlayers with 100 µs (red) and 1 µs update pulses.…”

Section: Results Of Multilayer Perceptron Simulations With Pcmo-based Memristive Switching Weightsmentioning

confidence: 99%

“…With an ON/OFF ratio of around 30, Wu et al (2020) showed an accuracy of about 95%. Even higher accuracies can be reached using memristive devices as storage for weights in a fashion of digital numbers instead of analog weights (Moon et al, 2015) (Babu et al, 2018).…”

Section: Results Of Multilayer Perceptron Simulations With Pcmo-based Memristive Switching Weightsmentioning

confidence: 99%

“…Previous works already show that memristive crossbar arrays allow for efficient vector-matrix multiplication (Cai et al, 2019). The use in ANNs was demonstrated on many network types such as single-layer perceptrons (Alibart et al, 2013;Prezioso et al, 2015) as well as multilayer perceptrons (Moon et al, 2015;Burr et al, 2017;Babu et al, 2018;Go et al, 2019;Wu et al, 2020) and convolutional neural networks (CNNs) (Yakopcic et al, 2017). Many groups show that memristive devices can already today replace conventional networks trained in software for many applications.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

et al. 2021

View full text Add to dashboard Cite

Memristive devices are novel electronic devices, which resistance can be tuned by an external voltage in a non-volatile way. Due to their analog resistive switching behavior, they are considered to emulate the behavior of synapses in neuronal networks. In this work, we investigate memristive devices based on the field-driven redox process between the p-conducting Pr0.7Ca0.3MnO3 (PCMO) and different tunnel barriers, namely, Al2O3, Ta2O5, and WO3. In contrast to the more common filamentary-type switching devices, the resistance range of these area-dependent switching devices can be adapted to the requirements of the surrounding circuit. We investigate the impact of the tunnel barrier layer on the switching performance including area scaling of the current and variability. Best performance with respect to the resistance window and the variability is observed for PCMO with a native Al2O3 tunnel oxide. For all different layer stacks, we demonstrate a spike timing dependent plasticity like behavior of the investigated PCMO cells. Furthermore, we can also tune the resistance in an analog fashion by repeated switching the device with voltage pulses of the same amplitude and polarity. Both measurements resemble the plasticity of biological synapses. We investigate in detail the impact of different pulse heights and pulse lengths on the shape of the stepwise SET and RESET curves. We use these measurements as input for the simulation of training and inference in a multilayer perceptron for pattern recognition, to show the use of PCMO-based ReRAM devices as weights in artificial neural networks which are trained by gradient descent methods. Based on this, we identify certain trends for the impact of the applied voltages and pulse length on the resulting shape of the measured curves and on the learning rate and accuracy of the multilayer perceptron.

show abstract

Section: Results Of Multilayer Perceptron Simulations With Pcmo-based Memristive Switching Weightsmentioning

confidence: 99%

Section: Results Of Multilayer Perceptron Simulations With Pcmo-based Memristive Switching Weightsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

et al. 2021

View full text Add to dashboard Cite

show abstract

“…53,54 . Unlike many prior proposals 42,43,51,52 , our approach is suitable for large-scale dot-product circuits and has no endurance restrictions for inference operation, which is typical for other proposals 44–50,55–57 . We experimentally verify stochastic dot-product circuits based on metal-oxide memristors and embedded floating-gate memories by implementing and testing Boltzmann machine networks with non-binary weights and hardware-injected noise.…”

Section: Introductionmentioning

confidence: 99%

Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization

2019

View full text Add to dashboard Cite

The key operation in stochastic neural networks, which have become the state-of-the-art approach for solving problems in machine learning, information theory, and statistics, is a stochastic dot-product. While there have been many demonstrations of dot-product circuits and, separately, of stochastic neurons, the efficient hardware implementation combining both functionalities is still missing. Here we report compact, fast, energy-efficient, and scalable stochastic dot-product circuits based on either passively integrated metal-oxide memristors or embedded floating-gate memories. The circuit’s high performance is due to mixed-signal implementation, while the efficient stochastic operation is achieved by utilizing circuit’s noise, intrinsic and/or extrinsic to the memory cell array. The dynamic scaling of weights, enabled by analog memory devices, allows for efficient realization of different annealing approaches to improve functionality. The proposed approach is experimentally verified for two representative applications, namely by implementing neural network for solving a four-node graph-partitioning problem, and a Boltzmann machine with 10-input and 8-hidden neurons.

show abstract

“…Moreover, memristors have very similar characteristics to the learning and memory processes of the human synapse. The resistance of memristors varies with the magnitude and duration of the applied voltage, which can mimic human synaptic plasticity in a single unit [15][16][17][18], while traditional electronic circuits require many electronic components and software programs to simulate a synapse.…”

Section: Introductionmentioning

confidence: 99%

Vacancy-induced resistive switching and synaptic behavior in flexible BST@Cf memristor crossbars

Wang

Yue

Jiang

et al. 2020

Ceramics International

View full text Add to dashboard Cite

In this study, carbon fibers (Cf) coated with Ba0.6Sr0.4TiO3 (BST) (BST@Cf) were prepared by magnetron sputtering and subsequently heated in nitrogen to produce oxygen vacancies. BST@Cf and nitrogen-treated BST@Cf were cross-stacked on polyimide (PI) film to make a BST@Cf memristor. The electrical 2 properties of BST@Cf memristor were measured after being bent 3000 times. The device exhibits bipolar figure-of-eight (f8) hysteresis loop characteristics under applied voltage. The hysteresis loops narrow with increasing temperature of heat treatment in nitrogen, due to decrease in oxygen vacancy concentration. The hysteresis loops demonstrate the switching process of resistance between high resistance state (HRS) and low resistance state (LRS), with a maximum HRS/LRS ratio of 10 6 . The switching process can be divided into two parts, corresponding to Schottky Emission and Fowler-Nordheim (F-N) Tunneling. It is notable that no electroforming voltage is required to stimulate the memristor. The constructed memristor was cycled successfully 1000 times and retained the LRS 787 s during power cut off. In addition, the device exhibited synaptic behavior including learning and forgetting processes, in accordance with the paired-pulse facilitation (PPF) rule.The use of BST@Cf in the construction of the nonvolatile memristor imparts flexibility to the device allowing for the possibility of wearable flexible intelligent memristor based electronic devices in the future.

show abstract

Stochastic learning in deep neural networks based on nanoscale PCMO device characteristics

Cited by 13 publications

References 24 publications

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization

Vacancy-induced resistive switching and synaptic behavior in flexible BST@Cf memristor crossbars

Contact Info

Product

Resources

About