Optimal Localist and Distributed Coding of Spatiotemporal Spike Patterns Through STDP and Coincidence Detection

Masquelier, Timothée; Kheradpisheh, Saeed Reza

doi:10.3389/fncom.2018.00074

Cited by 19 publications

(19 citation statements)

References 60 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, the STDP rule focuses on the first spikes of the input pattern which contain most of the information needed for pattern recognition. It has been shown that repeating spatio-temporal patterns can be detected and learned by a single neuron based on STDP [79], [80]. STDP can also solve difficult computational problems in localizing a repeating spatio-temporal spike pattern and enabling some forms of temporal coding, even if an explicit time reference is missing [79], [81].…”

Section: B Learning Rules In Snnsmentioning

confidence: 99%

Deep learning in spiking neural networks

et al. 2019

Self Cite

View full text Add to dashboard Cite

1 In recent years, deep learning has revolutionized the field of machine learning, for computer vision in particular. In this approach, a deep (multilayer) artificial neural network (ANN) is trained in a supervised manner using backpropagation. Vast amounts of labeled training examples are required, but the resulting classification accuracy is truly impressive, sometimes outperforming humans. Neurons in an ANN are characterized by a single, static, continuous-valued activation. Yet biological neurons use discrete spikes to compute and transmit information, and the spike times, in addition to the spike rates, matter. Spiking neural networks (SNNs) are thus more biologically realistic than ANNs, and arguably the only viable option if one wants to understand how the brain computes. SNNs are also more hardware friendly and energy-efficient than ANNs, and are thus appealing for technology, especially for portable devices. However, training deep SNNs remains a challenge. Spiking neurons' transfer function is usually non-differentiable, which prevents using backpropagation.Here we review recent supervised and unsupervised methods to train deep SNNs, and compare them in terms of accuracy, but also computational cost and hardware friendliness. The emerging picture is that SNNs still lag behind ANNs in terms of accuracy, but the gap is decreasing, and can even vanish on some tasks, while SNNs typically require many fewer operations.

show abstract

Section: B Learning Rules In Snnsmentioning

confidence: 99%

Deep learning in spiking neural networks

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…[83] Unsupervised learning helps to fully exploit the power advantage of SNN by localized training and STDP-based learning rules, [84,85] though it lacks in accuracy especially in the multilayer scenario. The merits of STDP of the localized spatio-temporal weights and sensitivity to repetitive spike trains in stochastic spike trains make it ideal for energy-efficient unsupervised learning, even with implicit time inference [86,87] when the training background is infused by noise. It has been proved that a full unsupervised SNN can achieve competitive training accuracy.…”

Section: Snn Training Topologiesmentioning

confidence: 99%

“…The full-digitalized RRAM crossbar exhibits great robustness when encountering computing variance caused by the stochastic switching property. [87] However, the intermediate states are Figure 6. Detailed structure of control bus and communication protocol.…”

Section: Cnn Circuitsmentioning

confidence: 99%

“…The full‐digitalized RRAM crossbar exhibits great robustness when encountering computing variance caused by the stochastic switching property. [ 87 ] However, the intermediate states are much more likely to encounter resistance variance caused by the inadequate conductive filament forming, thus suffering from precision loss in computation. With a fully digitalized computing interface, the requirement for AD/DA conversion in the traditional analogy RRAM crossbar is waived.…”

Section: Nn Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Low‐Power Computing with Neuromorphic Engineering

Liu

Chai

2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

Massive data centers and cloud computing infrastructures enable abundant data exchange over the Internet. As the demand for data transfer has increased, the number of data-generating devices and their power consumption have grown exponentially in the past few years. According to recent reports, the overall energy usage of information infrastructure in 2018 was estimated as 198 terawatt hours or almost 1% of demand for electricity in the world. [1] This ratio will continue to increase substantially in the era of abundant data, resulting in an unmanageable level of power consumption. [1] The overall power consumption of a computing system is determined by the data processing procedure as well as the power consumption of each single device and their density. The development of low-power devices has been overviewed in existing literatures by adopting either new device structures or new materials. [2-6] It has become more important to develop highly efficient processing units to minimize the overall power consumption of computing systems. Dennard predicted the reduction of switching power with the downward scaling of the feature size of the transistor, [7] and this trend ended around 2004. The operating voltage and current reduction of the transistor stagnated despite the evershrinking size of the transistor because of the increasing leakage current. The increase in operating frequency and the transistor density further exacerbates the power consumption of a computing system, which generates a large amount of heat and impedes power efficiency. Another challenge for power consumption is the intensive data transfer between data processing and memory units in conventional Von Neumann or Princeton computing architecture, the so-called "memory wall." [8] In this computing paradigm, the computing unit can only process one task at a certain interval and wait for memory to update its results, because both data and instructions are stored in the same memory space, which greatly limits the throughput and causes idle power consumption. Although mechanisms like cache and branch prediction can partially eliminate the issues, the "memory wall" still poses a grand challenge for massive data interchanging in modern processor technology. The prevalent synchronous clock design exacerbates the power problem, because the updates of all state-holding units are driven by a uniformly distributed global clock. Regardless of the usefulness of the task, all units are forced to respond to the arrival of clock edge. Thus, some unassigned units run idly and consume unnecessary power. In contrast to synchronous clock distribution, asynchronous distribution uses handshaking. The unparallelism issues also confine the power efficiency of high-performance computers. To fight against clock skew caused by uneven data transmission, a global clock with a high slew rate and elaborated frequency margin is required, which scarifies the area and power consumption. [9] As a result, a synchronous

show abstract

“…However, this method is still burdened with the training computational overhead and does little to utilize the efficiency of event driven computations. The SNN's Spike Time Dependent Plasticity (STDP) and spike-based back-propagation learning have been demonstrated to capture hierarchical features in SpikeCNNs (Masquelier and Thorpe, 2007;O'Connor et al, 2013;Panda et al, 2017;Kheradpisheh et al, 2018;Masquelier and Kheradpisheh, 2018;Falez et al, 2019). Both of these methods better equip the network to deal with event driven sensors, where the significant gains over CNNs could be realized.…”

Section: Introductionmentioning

confidence: 99%