An Ultra-Low Power Sigma-Delta Neuron Circuit

Abstract

Neural processing systems typically represent data using Leaky Integrate and Fire (LIF) neuron models that generate spikes or pulse trains at a rate proportional to their input amplitudes. This mechanism requires high firing rates when encoding time-varying signals, leading to increased power consumption. Neuromorphic systems that use adaptive LIF neuron models overcome this problem by encoding signals in the relative timing of their output spikes rather than their rate. In this paper, we analyze recent adaptive LIF neuron circuit implementations and highlight the analogies and differences between them and a first-order Σ∆ feedback loop. We propose a new Σ∆ neuron circuit that addresses some of the limitations in existing implementations and present simulation results that quantify the improvements. We show that the new circuit, implemented in a 1.8V , 180nm CMOS process, offers up to 42dB Signal to Distortion Ratio (SDR) and consumes orders of magnitude lower energy. Finally, we also demonstrate how the sigma-delta interpretation enables mapping of real-valued Recurrent Neural Networks (RNNs) to the spiking framework to emphasize the envisioned application of the proposed circuit.

Ultra-Low Power Silicon Neuron Circuit for Extreme-Edge Neuromorphic Intelligence

“…The neuron circuit presented has an energy per spike of tens of pJ for lower frequencies and pJ for higher frequencies, which is considerably lower compared to an analogous neuron design implemented in a 180 nm CMOS process [12]. Furthermore, it consumes less compared to a more recent design [34] at biologically plausible frequencies and it consumes one order of magnitude less compared to the state-of-the-art neuron circuit [35] at higher frequencies. We studied the mismatch sensitivity of the neuron circuit by performing Monte Carlo simulations and identified the parts of the circuit that are most critical to be optimized for variations, showing how the more sensitive sub-parts of the silicon neuron circuit are the LEAK block and the first part of the CC block.…”

“…Hence the differences reported here can be explained by the optimizations made at the circuit design level. The neuron circuit energy consumption at higher frequencies is compared with the Sigma-Delta neuron proposed in [35], which is one of the most recent mixed-signal silicon neuron circuit designs presented in the literature. Since the Sigma-Delta neuron presented in [35] was optimized for operation at higher frequencies in a range of 1 kHz to 10 MHz, we compare the energy per spike between these circuits in these ranges: the neuron proposed in this work consumes 1 pJ@2.1 kHz, approximately one order of magnitude less than the Sigma-Delta neuron (10 pJ).…”

Neuromorphic Systems Design by Matching Inductive Biases to Hardware Constraints

“…In a physical implementation, the fact that states of hidden neurons change slowly can be exploited by implementing them as leaky-integrate-and-fire (LIF) neurons with spike-frequency adaptation, which need to emit only few spikes to represent their state (Nair and Indiveri, 2019b). From the electrical engineering perspective, such neurons can be interpreted as -Modulators with unsigned steps (Yoon, 2016).…”

A 300mV Body-biased Silicon Neuron Circuit with High Robustness against Firing Frequency Variation

“…Moreover, traditional design methods result in bulky and high-power consumer neuromorphic circuits that fail to meet the requirements for the realization of large-scale spiking neural networks in neuromorphic processors [17][18]. Thus, improved mixed-signal and ultra-low voltage neurons circuit have been proposed to save areas and energy consumption [19][20][21]. Among the promising candidates, subthreshold design techniques reveal great potential to reduce current energy limitations [22][23].…”