Online Training of Spiking Recurrent Neural Networks with Phase-Change Memory Synapses

Demirag, Yigit; Frenkel, Charlotte; Payvand, Melika; Indiveri, Giacomo

doi:10.48550/arxiv.2108.01804

Cited by 2 publications

(2 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the presence of a teaching signal, a supervised learning framework can be used. For online and on-chip learning systems using events, this can be done through approximations of Backpropagation Through Time [42][43][44] , which can be implemented both on digital 45 , or in-memory memristive neuromorphic hardware 46 . However, in the absence of supervision, MEMSORN-like hardware changes its structure and self-organizes to cluster the input signal.…”

Section: Comparison To Other Neuromorphic Self-organizing Networkmentioning

confidence: 99%

Self-organization of an inhomogeneous memristive hardware for sequence learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

Learning is a fundamental component of creating intelligent machines. Biological intelligence orchestrates synaptic and neuronal learning at multiple time scales to self-organize populations of neurons for solving complex tasks. Inspired by this, we design and experimentally demonstrate an adaptive hardware architecture Memristive Self-organizing Spiking Recurrent Neural Network (MEMSORN). MEMSORN incorporates resistive memory (RRAM) in its synapses and neurons which configure their state based on Hebbian and Homeostatic plasticity respectively. For the first time, we derive these plasticity rules directly from the statistical measurements of our fabricated RRAM-based neurons and synapses. These "technologically plausible” learning rules exploit the intrinsic variability of the devices and improve the accuracy of the network on a sequence learning task by 30%. Finally, we compare the performance of MEMSORN to a fully-randomly-set-up spiking recurrent network on the same task, showing that self-organization improves the accuracy by more than 15%. This work demonstrates the importance of the device-circuit-algorithm co-design approach for implementing brain-inspired computing hardware.

show abstract

Section: Comparison To Other Neuromorphic Self-organizing Networkmentioning

confidence: 99%

Self-organization of an inhomogeneous memristive hardware for sequence learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…A simulation framework of differential-architecture crossbar arrays is developed in [27] to simulate spiking recurrent neural networks with PCM.…”

Section: A Related Workmentioning

confidence: 99%

A Software-Circuit-Device Co-Optimization Framework for Neuromorphic Inference Circuits

2022

View full text Add to dashboard Cite

Neuromorphic circuits, which usually use analog computation for vector-matrix multiplication (VMM) in neural networks (NN), are promising machine learning accelerators with much lower latency and power consumption than digital ones. Analog computation is expected to have a more efficient design space than digital computation since the signals are not digitized. Therefore, it is very suitable for Internet-of-Thing (IoT) applications that require ultra-low power consumption at a low cost. For IoT applications, sometimes it is also desirable to eliminate the digital circuits (such as adders, registers, shifters, multiplexers, and Analog-to-Digital Converters) between the VMM arrays to further reduce the power consumption. However, the optimization of a purely analog circuit is more difficult and requires full SPICE circuit simulations. In this paper, we present a software-circuit-device co-optimization framework using a python wrapper for automatic full circuit SPICE simulation and analysis for neuromorphic circuits. This framework allows users to experiment with how the NN design (software) affects the performance of the hardware neuromorphic circuits. It takes Verilog-A or SPICE models from calibrations or PDK in various technologies and emerging memories (such as ReRAM) without further calibration (unlike using behavior models). We show that the simulation time is reasonable even with hundreds of thousands of synapses under limited computation resources. Using ReRAM and a 45nm generic technology as an example, the effects of feedback network and OpAmp design, software ML architecture, and input data accuracy on the inference accuracy are studied.

show abstract

Online Training of Spiking Recurrent Neural Networks with Phase-Change Memory Synapses

Cited by 2 publications

References 48 publications

Self-organization of an inhomogeneous memristive hardware for sequence learning

Self-organization of an inhomogeneous memristive hardware for sequence learning

A Software-Circuit-Device Co-Optimization Framework for Neuromorphic Inference Circuits

Contact Info

Product

Resources

About