Dendritic-Inspired Processing Enables Bio-Plausible STDP in Compound Binary Synapses

Abstract

Brain-inspired learning mechanisms, e.g. spike timing dependent plasticity (STDP), enable agile and fast on-thefly adaptation capability in a spiking neural network. When incorporating emerging nanoscale resistive non-volatile memory (NVM) devices, with ultra-low power consumption and highdensity integration capability, a spiking neural network hardware would result in several orders of magnitude reduction in energy consumption at a very small form factor and potentially herald autonomous learning machines. However, actual memory devices have shown to be intrinsically binary with stochastic switching, and thus impede the realization of ideal STDP with continuous analog values. In this work, a dendritic-inspired processing architecture is proposed in addition to novel CMOS neuron circuits. The utilization of spike attenuations and delays transforms the traditionally undesired stochastic behavior of binary NVMs into a useful leverage that enables biologicallyplausible STDP learning. As a result, this work paves a pathway to adopt practical binary emerging NVM devices in brain-inspired neuromorphic computing.

Progress and Challenges of Neuroscience and Brain-inspired Artificial Intelligence

“…When incorporating nanoscale resistive non-volatile memory components (high-density integration capability and extremely low energy consumption), a hardware with SNNs will have several orders of reduction in energy consumption. A dendriticinspired processing architecture was presented in addition to complementary metal-oxide semiconductor (CMOS) neuron circuits (Wu and Saxena, 2018). Brain-inspired circuit design is thwarted by two limits: (1) understanding the event-driven spike processing of the human brain and (2) developing predictive models for the design and optimization of cognitive circuits.…”

Towards Neuromorphic Learning Machines Using Emerging Memory Devices with Brain-Like Energy Efficiency

“…Here, several (say M = 16) stochastic memristors were employed in parallel to obtain an approximate resolution of log 2 M = 4 bits on average. This concept was extended to include presynapstic axonal attenuation with parallel stochastic switching RRAMs [95,96]. Recently, the concept was further expanded to combine axonal (presynaptic) as well as dendritic (postsynaptic) processing [55].…”

“…Assuming Gaussian distribution of the program/erase threshold voltages, the stochastic switching behavior of the bistable RRAM device is given by cumulative probability p(V) = P(|V| > |V th +/− |) for a voltage drop of V across the device. This is expressed as [95,96]…”

“…Each dot in the plots represents the probability density of the particular ∆w transition between −16 and 16. With dendritic processing, a double exponential curve is fitted to the simulated STDP window with <1-unit fitting error; STDP window without dendrites has approximately 4-unit error when fitted to the double exponential [95,96]. Moreover, the axonal and dendrite coefficients, α i and β j , and potentially their respective time delays, can be customized to implement a wide range of STDP learning windows.…”

Feature Extraction using Spiking Convolutional Neural Networks

“…This is a new paradigm for implementing artificial neural networks using mechanisms that incorporate spike-timing dependent plasticity which is a learning algorithm discovered by neuroscientists [9] [21]. The promise of spiking networks is that they are less computationally intensive and much more energy efficient as the spiking algorithms can be implemented on a neuromorphic chip such as Intel's LOIHI chip [3] (operates at low power because it runs asynchronously using spikes) and other neuromorphic chips [40] [39] [41] [31]. Our work is based on the work of Masquelier and Thorpe [23] [22], and Kheradpisheh et al [14] [13].…”