Ultrahigh Density Memristor Neural Crossbar for On-Chip Supervised Learning

Chabi, Djaafar; Wang, Zhaohao; Bennett, Christopher H.; Klein, Jacques-Olivier; Zhao, Weisheng

doi:10.1109/tnano.2015.2448554

Cited by 43 publications

(16 citation statements)

References 33 publications

(64 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To show the former, we emulated a multi-layer perceptron system by feeding forward functions learned in a first layer to build the truth tables of linearly non-separable functions in subsequent layers 40 . Similarly to 44 , which showed a multi-layer memristive system can learn AND and NOT in a first layer with two SNUs and subsequently the 2-bit XOR function with another in the second, we learned the 3-bit XOR function (01101001). Three SNUs in the first layer and one SNU in the second (32 organic memristive devices total) are required to resolve this problem.…”

Section: Resultsmentioning

confidence: 99%

Physical Realization of a Supervised Learning System Built with Organic Memristive Synapses

Lin

Bennett

Cabaret

et al. 2016

Sci Rep

Self Cite

View full text Add to dashboard Cite

Multiple modern applications of electronics call for inexpensive chips that can perform complex operations on natural data with limited energy. A vision for accomplishing this is implementing hardware neural networks, which fuse computation and memory, with low cost organic electronics. A challenge, however, is the implementation of synapses (analog memories) composed of such materials. In this work, we introduce robust, fastly programmable, nonvolatile organic memristive nanodevices based on electrografted redox complexes that implement synapses thanks to a wide range of accessible intermediate conductivity states. We demonstrate experimentally an elementary neural network, capable of learning functions, which combines four pairs of organic memristors as synapses and conventional electronics as neurons. Our architecture is highly resilient to issues caused by imperfect devices. It tolerates inter-device variability and an adaptable learning rule offers immunity against asymmetries in device switching. Highly compliant with conventional fabrication processes, the system can be extended to larger computing systems capable of complex cognitive tasks, as demonstrated in complementary simulations.

show abstract

Section: Resultsmentioning

confidence: 99%

Physical Realization of a Supervised Learning System Built with Organic Memristive Synapses

Lin

Bennett

Cabaret

et al. 2016

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…Regarding the neuron designs, our NoProp digital design uses the simple, low-power CMOS inverter design proposed and successfully demonstrated in [62]. This system has a total energy footprint of less than 10 fJ per neuron.…”

Section: E Learning Energy Analysismentioning

confidence: 99%

Contrasting Advantages of Learning With Random Weights and Backpropagation in Non-Volatile Memory Neural Networks

et al. 2019

Self Cite

View full text Add to dashboard Cite

Recently, a Cambrian explosion of a novel, non-volatile memory (NVM) devices known as memristive devices have inspired effort in building hardware neural networks that learn like the brain. Early experimental prototypes built simple perceptrons from nanosynapses, and recently, fully-connected multilayer perceptron (MLP) learning systems have been realized. However, while backpropagating learning systems pair well with high-precision computer memories and achieve state-of-the-art performances, this typically comes with a massive energy budget. For future Internet of Things/peripheral use cases, system energy footprint will be a major constraint, and emerging NVM devices may fill the gap by sacrificing high bit precision for lower energy. In this paper, we contrast the well-known MLP approach with the extreme learning machine (ELM) or NoProp approach, which uses a large layer of random weights to improve the separability of high-dimensional tasks, and is usually considered inferior in a software context. However, we find that when taking the device non-linearity into account, NoProp manages to equal hardware MLP system in terms of accuracy. While also using a sign-based adaptation of the delta rule for energy-savings, we find that NoProp can learn effectively with four to six 'bits' of device analog capacity, while MLP requires eight-bit capacity with the same rule. This may allow the requirements for memristive devices to be relaxed in the context of online learning. By comparing the energy footprint of these systems for several candidate nanosynapses and realistic peripherals, we confirm that memristive NoProp systems save energy compared with MLP systems. Lastly, we show that ELM/NoProp systems can achieve better generalization abilities than nanosynaptic MLP systems when paired with pre-processing layers (which do not require backpropagated error). Collectively, these advantages make such systems worthy of consideration in future accelerators or embedded hardware. INDEX TERMS Hardware neural networks, memristive devices, online learning, edge computing.

show abstract

“…This scheme is generic to several classes of nanosynapses and highly resistant to device imperfections [13]. The training cell can use stateful memristive devices to correctly route programming pulses to output neurons with the corresponding error case in the readout crossbar layer [14]; the chosen nanosynapse has been directly integrated in such a scheme [15]. Fig.…”

Section: B Nanoelectric Implementationmentioning

confidence: 99%

Spatio-temporal learning with arrays of analog nanosynapses

Bennett

Querlioz

Klein

2017

2017 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH)

Self Cite

View full text Add to dashboard Cite

Emerging nanodevices such as resistive memories are being considered for hardware realizations of a variety of artificial neural networks (ANNs), including highly promising online variants of the learning approaches known as reservoir computing (RC) and the extreme learning machine (ELM). We propose an RC/ELM inspired learning system built with nanosynapses that performs both on-chip projection and regression operations. To address time-dynamic tasks, the hidden neurons of our system perform spatio-temporal integration and can be further enhanced with variable sampling or multiple activation windows. We detail the system and show its use in conjunction with a highly analog nanosynapse device on a standard task with intrinsic timing dynamics-the TI-46 battery of spoken digits. The system achieves nearly perfect (99%) accuracy at sufficient hidden layer size, which compares favorably with software results. In addition, the model is extended to a larger dataset, the MNIST database of handwritten digits. By translating the database into the time domain and using variable integration windows, up to 95% classification accuracy is achieved. In addition to an intrinsically low-power programming style, the proposed architecture learns very quickly and can easily be converted into a spiking system with negligible loss in performance-all features that confer significant energy efficiency.

show abstract

Ultrahigh Density Memristor Neural Crossbar for On-Chip Supervised Learning

Cited by 43 publications

References 33 publications

Physical Realization of a Supervised Learning System Built with Organic Memristive Synapses

Physical Realization of a Supervised Learning System Built with Organic Memristive Synapses

Contrasting Advantages of Learning With Random Weights and Backpropagation in Non-Volatile Memory Neural Networks

Spatio-temporal learning with arrays of analog nanosynapses

Contact Info

Product

Resources

About