Committee machines—a universal method to deal with non-idealities in memristor-based neural networks

Joksas, Dovydas; Freitas, Pedro; Chai, Zheng; Ng, Wing H.; Buckwell, M; Li, Chengzhu; Zhang, W.; Xia, Qiangfei; Kenyon, Anthony J.; Mehonić, Adnan

doi:10.1038/s41467-020-18098-0

Cited by 60 publications

(41 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A common approach exploits resistive memory crossbar arrays to implement in analog the vector matrix multiplication in a single step [45]. However, reliability issues affecting RRAM devices, such as the large cycle-to-cycle variability, limit the number of bits that can be reliably stored in a single device [49] thus suggesting that low-bit precision neural networks are a more suitable solution for the current state of the art RRAM devices. The extreme case of low-bit precision neural networks are BNNs, which have been shown to retain high classification accuracy despite the use of single-bit neuron weights and activations [5,50,51].…”

Section: Binarized Neural Network Applicationsmentioning

confidence: 99%

Multi-Input Logic-in-Memory for Ultra-Low Power Non-Von Neumann Computing

Zanotti¹,

Pavan²,

Puglisi³

2021

Micromachines

View full text Add to dashboard Cite

Logic-in-memory (LIM) circuits based on the material implication logic (IMPLY) and resistive random access memory (RRAM) technologies are a candidate solution for the development of ultra-low power non-von Neumann computing architectures. Such architectures could enable the energy-efficient implementation of hardware accelerators for novel edge computing paradigms such as binarized neural networks (BNNs) which rely on the execution of logic operations. In this work, we present the multi-input IMPLY operation implemented on a recently developed smart IMPLY architecture, SIMPLY, which improves the circuit reliability, reduces energy consumption, and breaks the strict design trade-offs of conventional architectures. We show that the generalization of the typical logic schemes used in LIM circuits to multi-input operations strongly reduces the execution time of complex functions needed for BNNs inference tasks (e.g., the 1-bit Full Addition, XNOR, Popcount). The performance of four different RRAM technologies is compared using circuit simulations leveraging a physics-based RRAM compact model. The proposed solution approaches the performance of its CMOS equivalent while bypassing the von Neumann bottleneck, which gives a huge improvement in bit error rate (by a factor of at least 108) and energy-delay product (projected up to a factor of 1010).

show abstract

Section: Binarized Neural Network Applicationsmentioning

confidence: 99%

Multi-Input Logic-in-Memory for Ultra-Low Power Non-Von Neumann Computing

Zanotti¹,

Pavan²,

Puglisi³

2021

Micromachines

View full text Add to dashboard Cite

show abstract

“…Then we calculate the mean of the results. Joksas et al did this by applying the committee machine theory into the in-memory computing devices [30]. Wan et al optimized this process by running a single model on the same device and reading the memory cells multiple times [31].…”

Section: Related Workmentioning

confidence: 99%

Optimizing for In-memory Deep Learning with Emerging Memory Technology

Wang¹,

Luo²,

Goh³

et al. 2021

Preprint

View full text Add to dashboard Cite

In-memory deep learning computes neural network models where they are stored, thus avoiding long distance communication between memory and computation units, resulting in considerable savings in energy and time. In-memory deep learning has already demonstrated orders of magnitude higher performance density and energy efficiency. The use of emerging memory technology promises to increase the gains in density, energy, and performance even further. However, emerging memory technology is intrinsically unstable, resulting in random fluctuations of data reads. This can translate to non-negligible accuracy loss, potentially nullifying the gains. In this paper, we propose three optimization techniques that can mathematically overcome the instability problem of emerging memory technology. They can improve the accuracy of the in-memory deep learning model while maximizing its energy efficiency. Experiments show that our solution can fully recover most models' state-of-the-art accuracy, and achieves at least an order of magnitude higher energy efficiency than the state-of-the-art.

show abstract

“…Due to the limitations of the Von Neumann architecture, the traditional storage-separated computing approach can no longer meet the fast-growing demand for big data computing [1][2][3]. Recently, people have started to turn their attention to storageintegrated bio-neural computing due to that there are 10 11 neurons and 10 15 synapses in the human brain, which can integrate complex cognitive functions such as perception, calculation, logic, language, vision, and memory [4][5][6].…”

Section: Introductionmentioning

confidence: 99%

Memristor based on α-In2Se3 for emulating biological synaptic plasticity and learning behavior

Zhao

Pei

Zhang³

et al. 2022

Sci. China Mater.

View full text Add to dashboard Cite

Nowadays, memristors are extremely similar to biological synapses and can achieve many basic functions of biological synapses, making them become a new generation of research hotspots for brain-like neurocomputing. In this work, we prepare a memristor based on two-dimensional α-In 2 Se 3 nanosheets, which exhibits excellent electrical properties, faster switching speeds, and continuous tunability of device conduction. Meanwhile, most basic bio-synapse functions can be implemented faithfully, such as short-term memory (STM), long-term memory (LTM), four different types of spike-timing-dependent plasticity (STDP), and paired-pulse facilitation (PPF). More importantly, we systematically study three effective methods to achieve LTM, in which the reinforcement learning can be faithfully simulated according to the Ebbinghaus forgetting curve. Therefore, we believe this work will promote the development of learning functions for brain-like computing and artificial intelligence.

show abstract

Committee machines—a universal method to deal with non-idealities in memristor-based neural networks

Cited by 60 publications

References 32 publications

Multi-Input Logic-in-Memory for Ultra-Low Power Non-Von Neumann Computing

Multi-Input Logic-in-Memory for Ultra-Low Power Non-Von Neumann Computing

Optimizing for In-memory Deep Learning with Emerging Memory Technology

Memristor based on α-In2Se3 for emulating biological synaptic plasticity and learning behavior

Contact Info

Product

Resources

About