PCM-Trace: Scalable Synaptic Eligibility Traces with Resistivity Drift of Phase-Change Materials

Abstract

Dedicated hardware implementations of spiking neural networks that combine the advantages of mixed-signal neuromorphic circuits with those of emerging memory technologies have the potential of enabling ultra-low power pervasive sensory processing. To endow these systems with additional flexibility and the ability to learn to solve specific tasks, it is important to develop appropriate on-chip learning mechanisms. Recently, a new class of three-factor spike-based learning rules have been proposed that can solve the temporal credit assignment problem and approximate the error back-propagation algorithm on complex tasks. However, the efficient implementation of these rules on hybrid CMOS/memristive architectures is still an open challenge. Here we present a new neuromorphic building block, called PCM-trace, which exploits the drift behavior of phasechange materials to implement long lasting eligibility traces, a critical ingredient of three-factor learning rules. We demonstrate how the proposed approach improves the area efficiency by > 10× compared to existing solutions and demonstrates a technologically plausible learning algorithm supported by experimental data from device measurements.

Online Training of Spiking Recurrent Neural Networks with Phase-Change Memory Synapses

“…Our work compares several previously developed methods that are designed to cope with memristor non-idealities and demonstrates that accumulating gradients allows more reliable programming of PCM devices, reduces the number of programming devices and outperforms other synaptic weight-update mechanisms. Future work will need to evaluate the impacts of the implemented weight update schemes using more extensive datasets for a more interpretable benchmarking, further incorporate the PCM devices for emulating temporal dynamics such as eligibility traces [27], as well as exploring other learning rules such as OSTL or Real Time Recurrent Learning (RTRL) [73].…”

“…Their tiny footprint, fast read/write operation and multi-bit storage capacity make PCMs an ideal candidate for implementing in-memory computation of synaptic propagation [23,24]. Consequently, there has been an increased interest for the employment of PCM technology in neuromorphic computing applications [11,25,26,27].…”

Bottom-Up and Top-Down Neural Processing Systems Design: Neuromorphic Intelligence as the Convergence of Natural and Artificial Intelligence

“…Beyond the usual prospects for improvement in density and power efficiency linked with inmemory computation, memristors offer specific synergies for neuromorphic engineering, such as characteristics similar to those of biological synapses [89]. Furthermore, a neuromorphic approach exploiting non-idealities instead of mitigating them could be particularly appropriate to alleviate the high levels of noise and mismatch encountered in these devices [86], or to leverage parasitic effects such as the conductance drift [90]. However, high-yield large-scale co-integration with CMOS is still at an early stage [91], [92].…”

“…Furthermore, as the global modulation signal may be delayed over second-long behavioral timescales, there is a need for synapses to maintain a memory of their past activity, which may be achieved through local synaptic eligibility traces [169]. While the computation of eligibility traces is already supported by some neuromorphic platforms with the help of von-Neumann co-processors [71], [74], [170], a fully-parallel implementation was proposed in [90] by exploiting the drift non-ideality of phase change memory (PCM) devices. This growing complexity in synaptic learning rules is also closely related to dendritic computation (Section III-A3).…”

Palimpsest Memories Stored in Memristive Synapses

“…The flexibility promised by dynamic memory consolidation has naturally attracted the attention of AI hardware design and in particular that of memristive technologies which have already showcased their potential in numerous neuromorphic applications [Stathopoulos et al, 2017, Gupta et al, 2016, Yoon et al, 2018, Boybat et al, 2018. Memristors have been used to implement metaplasticity, i.e tuning of the learning rate Bear, 1996, Fusi et al, 2005], on CMOS-based artificial synapses in spiking neural networks (SNNs) [Brivio et al, 2019, Demirag et al, 2021. In a similar vein, non-volatile resistive RAM (RRAM) synapses use explicitly modulated bias voltage to tune their switching (i.e.…”