A 0.086-mm<formula> <tex>$^2$</tex> </formula> 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS

Abstract

Shifting computing architectures from von Neumann to event-based spiking neural networks (SNNs) uncovers new opportunities for low-power processing of sensory data in applications such as vision or sensorimotor control. Exploring roads toward cognitive SNNs requires the design of compact, low-power and versatile experimentation platforms with the key requirement of online learning in order to adapt and learn new features in uncontrolled environments. However, embedding online learning in SNNs is currently hindered by high incurred complexity and area overheads. In this work, we present ODIN, a 0.086-mm 2 64ksynapse 256-neuron online-learning digital spiking neuromorphic processor in 28nm FDSOI CMOS achieving a minimum energy per synaptic operation (SOP) of 12.7pJ. It leverages an efficient implementation of the spike-driven synaptic plasticity (SDSP) learning rule for high-density embedded online learning with only 0.68µm 2 per 4-bit synapse. Neurons can be independently configured as a standard leaky integrate-and-fire (LIF) model or as a custom phenomenological model that emulates the 20 Izhikevich behaviors found in biological spiking neurons. Using a single presentation of 6k 16×16 MNIST training images to a single-layer fully-connected 10-neuron network with on-chip SDSP-based learning, ODIN achieves a classification accuracy of 84.5% while consuming only 15nJ/inference at 0.55V using rank order coding. ODIN thus enables further developments toward cognitive neuromorphic devices for low-power, adaptive and lowcost processing.Index Terms-Neuromorphic engineering, spiking neural networks, synaptic plasticity, online learning, Izhikevich behaviors, phenomenological modeling, event-based processing, CMOS digital integrated circuits, low-power design.

Digital multiplier‐less implementation of high‐precision SDSP and synaptic strength‐based STDP

“…This rule is also competitive among the state-of-the-art spike-based machine learning methods. 1,10,43 According to this rule, every time the post-synaptic neuron emits a spike, an internal variable Ca 2+ , which represents calcium concentration and is proportional to the neuron's recent spiking activity, is incremented by a value J c and then decays with a time constant Ca , according to the dynamics of Equation (1),…”

“…1,7,8 Several research groups around the world are developing custom hardware systems for simulating large-scale neural models. 1,9,10 So far, these systems have limitations in applications that require accurate calculations. For instance, simulating networks with reinforcement learning capabilities 11 or convolutional spiking neural networks for pattern recognition rely on precise values of weights.…”

An Ultra-Low Power Sigma-Delta Neuron Circuit

“…The mixed-signal neuron designs in BrainScaleS [20] and Neurogrid [3] offer the closest comparison to this work. TrueNorth [21], Loihi [22] and ODIN [23] are digital systems that use advanced processes, time-multiplexing of neurons and low supply voltages. They are also designed for high input and output data rates and are arguably less suited than the proposed neuron design for low-bandwidth sensor data processing applications.…”

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation

“…Communication of forward and backward propagated information is fully based on signed binary events, without the need to process floating point numbers, in contrast to the standard backpropagation algorithm [9]. This makes it particularly suitable for integration in digital neuromorphic platforms (such as [10]).…”