Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms

Abstract

Recurrent neural networks are currently subject to intensive research efforts to solve temporal computing problems. Neuromorphic processors (NPs), composed of networked neuron and synapse circuit models, natively compute in time and offer an ultralow power solution particularly suited to emerging temporal edge-computing applications (wearable medical devices, for example). The most significant roadblock to addressing useful problems with neuromorphic hardware is the difficulty in maintaining healthy network dynamics in recurrent neural networks. In animal nervous systems, this is achieved via a multitude of adaptive homeostatic mechanisms which act over multiple time scales to counteract network instability induced via drift, component failure, or learning processes such as spike-timing dependent plasticity. One such mechanism is neuronal intrinsic plasticity (IP) where a neuron adapts its parameters which govern its excitability to fire around a target rate. The approach employed in state of the art NPs, based on a central volatile memory remotely setting model parameters, critically constrains parameter variety and bandwidth rendering realization of these essential mechanisms impossible. This paper demonstrates how reconfigurable nonvolatile resistive memories can be incorporated into neuron and synapse circuits allowing memory to be truly colocalized with the computational units in the computing fabric and facilitating the realization of massively parallel local plasticity mechanisms in neuromorphic hardware. Exploiting nonconventional programming operations of HfO2 based RRAM (stochastic SET and the RESET random variable), we propose a technologically plausible IP algorithm and demonstrate its use in the case of a recurrent neural network topology whereby the system self-organizes to sustain stable and healthy network dynamics around a target firing rate.

Analysis of the Voltage-Dependent Plasticity in Organic Neuromorphic Devices

“…where T p is the pulse time period and τ d is the detrapping time constant. Merging Equations (1) and (2) gives the final model…”

“…Neuromorphic engineering is an emerging technological area, which aims at mimicking the biological functionalities of neurons, synapses, or a whole brain by various electronic materials and devices [1][2][3][4][5][6]. Recently, the use of organic electronics in neuromorphic systems has gained tremendous attention, thanks to its capacity to expand the technological scope of such systems by creating unconventional interfaces such as direct neuroprotheses and robotic sensory bridges [7][8][9][10].…”

Bio-Inspired Architectures Substantially Reduce the Memory Requirements of Neural Network Models

“…It is interesting to note that these design choices also result in a technologically-plausible model that naturally lends itself to a future silicon neuromorphic implementation. The required hyperbolic tangent, sigmoid and leaky-integration functions are readily implemented using numerous analogue or digital silicon circuits (Lansner and Lehmann, 1993 ; Indiveri et al, 2011 ; Davies et al, 2018 ; Dalgaty et al, 2019 ). Similarly, the event-based input generated by the models sensory neurons can be realized through the use of delta-modulator circuits (Corradi and Indiveri, 2015 ) and the inputs of these delta-modulator circuits could be provided by existing bio-mimetic micro-electro-mechanical systems (MEMS) implementations of cricket filiform hairs (Krijnen et al, 2006 ).…”

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

“…The integration of CMOS technology with that of the emerging devices has been demonstrated for non-volatile filamentary switches [147] already at a commercial level [148]. There have also been some efforts in combining CMOS and memristor technologies to design supervised local error-based learning circuits using only one network layer by exploiting the properties of memristive devices [143], [149], [150].…”