Activity-difference training of deep neural networks using memristor crossbars

Yi, Su-in; Kendall, Jack D.; Williams, R. Stanley; Kumar, Suhas

doi:10.1038/s41928-022-00869-w

Cited by 30 publications

(29 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent techniques show overcoming memristor device nonidealities like faulty devices, device-to-device variability, random telegraph noise and line resistance by ensemble averaging, 370 and incorporating noise during online training for better performance of large-scale memristor based neural networks. 215,225,371 Current approaches to neuromorphic architectures focus primarily on a bottom-up approach to codesign, which does not effectively exploit the physics of the devices for computation. The challenge is that typically, low-level device engineers are left with the burden of demonstrating the usability of their device.…”

Section: Analog Circuit Designmentioning

confidence: 99%

“…Appropriate algorithms are required to exploit the noise in memristor devices for robust computation. Recent techniques show overcoming memristor device nonidealities like faulty devices, device-to-device variability, random telegraph noise and line resistance by ensemble averaging, and incorporating noise during online training for better performance of large-scale memristor based neural networks. ,, …”

Section: Other Approaches Of Memristive Technologymentioning

confidence: 99%

See 1 more Smart Citation

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems

et al. 2023

View full text Add to dashboard Cite

Memristive technology has been rapidly emerging as a potential alternative to traditional CMOS technology, which is facing fundamental limitations in its development. Since oxide-based resistive switches were demonstrated as memristors in 2008, memristive devices have garnered significant attention due to their biomimetic memory properties, which promise to significantly improve power consumption in computing applications. Here, we provide a comprehensive overview of recent advances in memristive technology, including memristive devices, theory, algorithms, architectures, and systems. In addition, we discuss research directions for various applications of memristive technology including hardware accelerators for artificial intelligence, in-sensor computing, and probabilistic computing. Finally, we provide a forward-looking perspective on the future of memristive technology, outlining the challenges and opportunities for further research and innovation in this field. By providing an up-to-date overview of the state-of-the-art in memristive technology, this review aims to inform and inspire further research in this field.

show abstract

Section: Analog Circuit Designmentioning

confidence: 99%

Section: Other Approaches Of Memristive Technologymentioning

confidence: 99%

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Kumar's group reported on the preparation of a memristor array using tantalum oxide as a functional layer. 51 They used an effective method to constrain the memristor network and reduce the differences in the execution of calculations by the array, thereby significantly improving the number of bits used for computing and the energy efficiency of the device. In particular, the device can be used to effectively classify and organize Braille by using simulated neural networks.…”

Section: Memristor-based Chipsmentioning

confidence: 99%

Section: Memristor-based Chipsmentioning

confidence: 99%

Memristor-Based Artificial Chips

Sun,

Chen,

Zhou

et al. 2023

ACS Nano

View full text Add to dashboard Cite

Memristors, promising nanoelectronic devices with in-memory resistive switching behavior that is assembled with a physically integrated core processing unit (CPU) and memory unit and even possesses highly possible multistate electrical behavior, could avoid the von Neumann bottleneck of traditional computing devices and show a highly efficient ability of parallel computation and high information storage. These advantages position them as potential candidates for future data-centric computing requirements and add remarkable vigor to the research of next-generation artificial intelligence (AI) systems, particularly those that involve brain-like intelligence applications. This work provides an overview of the evolution of memristor-based devices, from their initial use in creating artificial synapses and neural networks to their application in developing advanced AI systems and brain-like chips. It offers a broad perspective of the key device primitives enabling their special applications from the view of materials, nanostructure, and mechanism models. We highlight these demonstrations of memristor-based nanoelectronic devices that have potential for use in the field of brain-like AI, point out the existing challenges of memristor-based nanodevices toward brain-like chips, and propose the guiding principle and promising outlook for future device promotion and system optimization in the biomedical AI field.

show abstract

“…This suggests the existence of biological principles accommodating that, which we need to understand and port to successfully implement neuromorphic hardware. The spiking TM and other brain-inspired self-organizing networks (Lazar et al 2009, Yi et al 2022 suggest a set of biological concepts that might be at the heart of brain processing capabilities. For instance, the highly sparse connectivity and activity of the spiking TM are observed in biological networks, and they are essential for increasing the capacity of the system and decreasing energy consumption.…”

Section: Relationship To Previous Modelsmentioning

confidence: 99%

Sequence learning in a spiking neuronal network with memristive synapses

Bouhadjar,

Siegel,

Tetzlaff

et al. 2023

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience, but also to pave the way to new technological brain-inspired applications. A previously developed spiking neural network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. An emerging type of hardware that may efficiently run this type of algorithm is neuromorphic hardware. It emulates the way the brain processes information and maps neurons and synapses directly into a physical substrate. Memristive devices have been identified as potential synaptic elements in neuromorphic hardware. In particular, redox-induced resistive random access memories (ReRAM) devices stand out at many aspects. They permit scalability, are energy efficient and fast, and can implement biological plasticity rules. In this work, we study the feasibility of using ReRAM devices as a replacement of the biological synapses in the sequence learning model. We implement and simulate the model including the ReRAM plasticity using the neural network simulator NEST. We investigate two types of ReRAM memristive devices: (i) a gradual, analog switching device, and (ii) an abrupt, binary switching device. We study the effect of different device properties on the performance characteristics of the sequence learning model, and demonstrate that, in contrast to many other artificial neural networks, this architecture is resilient with respect to changes in the on-off ratio and the conductance resolution, device variability, and device failure.

show abstract

Activity-difference training of deep neural networks using memristor crossbars

Cited by 30 publications

References 40 publications

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems

Memristor-Based Artificial Chips

Sequence learning in a spiking neuronal network with memristive synapses

Contact Info

Product

Resources

About