Gradient-descent-based learning in memristive crossbar arrays

Nair, Manu V; Dudek, Piotr

doi:10.1109/ijcnn.2015.7280658

Cited by 25 publications

(16 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The key issue addressed in our work is how to use the plasticity effects in synapses represented by memristors with multiple resistive states to locally implement the learning rule. The main distinction between our results and the related studies [25] is that we have implemented the mechanism, which is able to propagate error backwards and is needed for multi-layered networks. This is important for the deep learning schemes.…”

Section: Discussionmentioning

confidence: 90%

“…This is important for the deep learning schemes. Note that in [25] the single layer perceptron is considered, and the method proposed in that work cannot be used to propagate the error between the layers. The implementation of the learning rule requires the conversion of signals x i and δ j at opposite electrodes of a memristor in a crossbar to the voltage drop across the crossbar that would change the memristor conductivity proportional to the product x i × δ j .…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

An approximate backpropagation learning rule for memristor based neural networks using synaptic plasticity

et al. 2017

View full text Add to dashboard Cite

We describe an approximation to backpropagation algorithm for training deep neural networks, which is designed to work with synapses implemented with memristors. The key idea is to represent the values of both the input signal and the backpropagated delta value with a series of pulses that trigger multiple positive or negative updates of the synaptic weight, and to use the min operation instead of the product of the two signals. In computational simulations, we show that the proposed approximation to backpropagation is well converged and may be suitable for memristor implementations of multilayer neural networks.

show abstract

Section: Discussionmentioning

confidence: 90%

Section: Discussionmentioning

confidence: 99%

An approximate backpropagation learning rule for memristor based neural networks using synaptic plasticity

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Furthermore, it will be much more difficult if the electronic synapse devices have nonlinear conductance responses. In other words, the use of multiple pulses for precise weight updates is impractical in actual electronic devices (Nair and Dudek, 2015). Therefore, we propose a weight-updating method based on a BP algorithm for HW-DNNs in which the amount of the weight change (equivalently, the learning rate in SW-DNNs) is determined by the amount of the conductance change of the electronic synapse devices.…”

Section: Weight-updating Methodsmentioning

confidence: 99%

“…Recently, several types of emerging electronic synapse devices such as phase change memory (PCRAM) (Suri et al, 2011;Wright et al, 2011;Kuzum et al, 2012), resistive change memory (RRAM) (Jo et al, 2010;Ohno et al, 2011;Wu et al, 2012;Yu et al, 2012), ferroelectric devices (Chanthbouala et al, 2012), and FET-based devices (Diorio et al, 1996;Ziegler et al, 2012;Kim et al, 2016) have been proposed to mimic biological synapses. Although most of these works have focused on spike-timingdependent-plasticity (STDP, (Bi and Poo, 1998)) learning algorithm (Kuzum et al, 2013), the learning performance using STDP is still in its early stage (Burr et al, 2014;Nair and Dudek, 2015). Unlike the approach in which STDP is used, electronic synapse devices (Burr et al, 2014;Prezioso et al, 2015;Merrikh-Bayat et al, 2015) can also be applied to deep neural networks (DNNs) with well-studied back-propagation (BP) algorithms (Rumelhart et al, 1986).…”

Section: Introductionmentioning

confidence: 99%

Adaptive learning rule for hardware-based deep neural networks using electronic synapse devices

Lim¹,

Bae²,

Eum³

et al. 2018

Neural Comput & Applic

View full text Add to dashboard Cite

In this paper, we propose a learning rule based on a back-propagation (BP) algorithm that can be applied to a hardware-based deep neural network (HW-DNN) using electronic devices that exhibit discrete and limited conductance characteristics. This adaptive learning rule, which enables forward, backward propagation, as well as weight updates in hardware, is helpful during the implementation of power-efficient and high-speed deep neural networks. In simulations using a three-layer perceptron network, we evaluate the learning performance according to various conductance responses of electronic synapse devices and weight-updating methods. It is shown that the learning accuracy is comparable to that obtained when using a software-based BP algorithm when the electronic synapse device has a linear conductance response with a high dynamic range. Furthermore, the proposed unidirectional weight-updating method is suitable for electronic synapse devices which have nonlinear and finite conductance responses. Because this weight-updating method can compensate the demerit of asymmetric weight updates, we can obtain better accuracy compared to other methods. This adaptive learning rule, which can be applied to full hardware implementation, can also compensate the degradation of learning accuracy due to the probable device-to-device variation in an actual electronic synapse device.

show abstract

“…BL is based on the Random Vector Functional-link Neural Network (RVFLNN) previously proposed in [32]. Instead of gradient-descent-based learning algorithms [33], RVFLNN provides the generalization capability as a function approximation by calculating the pseudoinverse to find the desired connection weights. However, RVLNN does not work well in the modern large data era.…”

Section: Bl Algorithmmentioning

confidence: 99%

Identification of Coal Geographical Origin Using Near Infrared Sensor Based on Broad Learning

Lei

Rao

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

Geographical origin, an important indicator of the chemical composition and quality grading, is one essential factor that should be taken into account in evaluating coal quality. However, traditional coal origin identification methods based on chemistry experiments are not only time consuming and labour intensive, but also costly. Near-Infrared (NIR) spectroscopy is an effective and efficient way to measure the chemical compositions of samples and has demonstrated excellent performance in various fields of quantitative and qualitative research. In this study, we employ NIR spectroscopy to identify coal origin. Considering the fact that the NIR spectra of coal samples always contain a large amount of redundant information and the number of samples is small, the broad learning algorithm is utilized here as the modelling system to classify the coal geographical origin. In addition, the particle swarm optimization algorithm is introduced to improve the structure of the Broad Learning (BL) model. We compare the improved model with the other five multivariate classification methods on a dataset with 243 coal samples collected from five countries. The experimental results indicate that the improved BL model can achieve the highest overall accuracy of 97.05%. The results obtained in this study suggest that the NIR technique combined with machine learning methods has significant potential for further development of coal geographical origin identification systems.

show abstract

Gradient-descent-based learning in memristive crossbar arrays

Cited by 25 publications

References 18 publications

An approximate backpropagation learning rule for memristor based neural networks using synaptic plasticity

An approximate backpropagation learning rule for memristor based neural networks using synaptic plasticity

Adaptive learning rule for hardware-based deep neural networks using electronic synapse devices

Identification of Coal Geographical Origin Using Near Infrared Sensor Based on Broad Learning

Contact Info

Product

Resources

About