The impact of on-chip communication on memory technologies for neuromorphic systems

Moradi, Saber; Manohar, Rajit

doi:10.1088/1361-6463/aae641

Cited by 24 publications

(12 citation statements)

References 76 publications

(120 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The energy estimates were obtained by dividing the total energy consumption by computation, routing, and static energy, and then calculating each energy proportionally to the number of spikes, Figure 5: Firing rate-regularity graph for various neural coding schemes spiking density, and latency, respectively. We referred to [6,7,26] for the energy consumption ratios of the three parts (computation, routing, and static), and the estimated values were normalized for each dataset. As a result of our estimation, our method consumed the least energy in almost all cases with higher accuracy.…”

Section: Comparison With Other Deep Snn Algorithmsmentioning

confidence: 99%

Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks

Park

Kim

Choe

et al. 2019

Proceedings of the 56th Annual Design Automation Conference 2019

View full text Add to dashboard Cite

The spiking neural networks (SNNs) are considered as one of the most promising artificial neural networks due to their energyefficient computing capability. Recently, conversion of a trained deep neural network to an SNN has improved the accuracy of deep SNNs. However, most of the previous studies have not achieved satisfactory results in terms of inference speed and energy efficiency. In this paper, we propose a fast and energy-efficient information transmission method with burst spikes and hybrid neural coding scheme in deep SNNs. Our experimental results showed the proposed methods can improve inference energy efficiency and shorten the latency.

show abstract

Section: Comparison With Other Deep Snn Algorithmsmentioning

confidence: 99%

Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks

Park

Kim

Choe

et al. 2019

Proceedings of the 56th Annual Design Automation Conference 2019

View full text Add to dashboard Cite

show abstract

“…How spike trains encode information is among the most important questions in neuroscience [730] . Both, spike timing and spike frequency have been proposed as modes of information transfer in biological brains [856] and both can codes can be investigated in oxide-based synaptors, neuristors and arrays [857] , [858] , [859] , [860] . Owing to its enhanced energy efficiency, spike timing appears to be a preponderant code in biological brains while frequency codes are believed to be exponentially more costly [861] , [862] , [863] .…”

Section: Neural Codesmentioning

confidence: 99%

Functional Oxides for Photoneuromorphic Engineering: Toward a Solar Brain

Pérez‐Tomás

2019

Adv Materials Inter

View full text Add to dashboard Cite

New device concepts and new computing principles are needed to balance our ever-growing appetite for data and information with the realization of the goals of increased energy efficiency, reduction in CO 2 emissions and the circular economy. Neuromorphic or synaptic electronics is an emerging field of research aiming to overcome the current computer's Von-Neumann bottleneck by building artificial neuronal systems to mimic the extremely energy efficient biological synapses. The introduction of photovoltaic and/or photonic aspects into these neuromorphic architectures will produce self-powered adaptive electronics but may also open up new possibilities in artificial neuroscience, artificial neural communications, sensing and machine learning which would enable, in turn, a new era for computational systems owing to the possibility of attaining high bandwidths with much reduced power consumption. This perspective is focused on recent progress in the implementation of functional oxide thinfilms into photovoltaic and neuromorphic applications towards the envisioned goal of selfpowered photovoltaic neuromorphic systems or a solar brain.

show abstract

“…What is worse, bus arbitration, addressing and latency prolong the transfer time (and decrease the system's efficacy). This type of communicational burst may easily lead to a ''communicational collapse'' [31], but it may also produce unintentional ''neuronal avalanches'' [5].…”

Section: Using High-speed Bus(es)mentioning

confidence: 99%

Which scaling rule applies to large artificial neural networks

Végh¹

2021

Neural Comput & Applic

View full text Add to dashboard Cite

Experience shows that cooperating and communicating computing systems, comprising segregated single processors, have severe performance limitations, which cannot be explained using von Neumann’s classic computing paradigm. In his classic “First Draft,” he warned that using a “too fast processor” vitiates his simple “procedure” (but not his computing model!); furthermore, that using the classic computing paradigm for imitating neuronal operations is unsound. Amdahl added that large machines, comprising many processors, have an inherent disadvantage. Given that artificial neural network’s (ANN’s) components are heavily communicating with each other, they are built from a large number of components designed/fabricated for use in conventional computing, furthermore they attempt to mimic biological operation using improper technological solutions, and their achievable payload computing performance is conceptually modest. The type of workload that artificial intelligence-based systems generate leads to an exceptionally low payload computational performance, and their design/technology limits their size to just above the “toy” level systems: The scaling of processor-based ANN systems is strongly nonlinear. Given the proliferation and growing size of ANN systems, we suggest ideas to estimate in advance the efficiency of the device or application. The wealth of ANN implementations and the proprietary technical data do not enable more. Through analyzing published measurements, we provide evidence that the role of data transfer time drastically influences both ANNs performance and feasibility. It is discussed how some major theoretical limiting factors, ANN’s layer structure and their methods of technical implementation of communication affect their efficiency. The paper starts from von Neumann’s original model, without neglecting the transfer time apart from processing time, and derives an appropriate interpretation and handling for Amdahl’s law. It shows that, in that interpretation, Amdahl’s law correctly describes ANNs.

show abstract

The impact of on-chip communication on memory technologies for neuromorphic systems

Cited by 24 publications

References 76 publications

Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks

Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks

Functional Oxides for Photoneuromorphic Engineering: Toward a Solar Brain

Which scaling rule applies to large artificial neural networks

Contact Info

Product

Resources

About