Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization

Mahmoodi, Mohammad Reza; Prezioso, M.; Strukov, Dmitri B.

doi:10.1038/s41467-019-13103-7

Cited by 80 publications

(67 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If these weights are set as nonzero, it can be assumed as a self feedback. If they are set properly, the network would change into a chaotic neural network that can solve combinatorial optimization problems, namely, to find the global minimum instead of nearest minimum. Recently, this has also been realized utilizing the random telegraph noise in memristive devices .…”

Section: Resultsmentioning

confidence: 99%

Memristor‐Based Biologically Plausible Memory Based on Discrete and Continuous Attractor Networks for Neuromorphic Systems

Wang

Liu

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

To approach an advanced neuromorphic system, a significant unsettled problem is how to realize biologically plausible memory structures that are dramatically different from classical computers. Herein, a physical system based on memristors is simulated to realize associative memory based on discrete attractor networks, which is essentially content‐based storage, and the influence of device characteristics on network performance is systematically studied. An in situ unsupervised learning method is applied to make greater use of array structure and competitions between neurons, demonstrating significant performance improvement in memory capacity and noise tolerance compared with existing supervised approaches. By extending to continuous attractor neural networks (CANNs), working memory is realized based on memristors for the first time via simulation, and the write and read noises in memristor arrays are found to have different impacts on the ability of CANN in maintaining dynamic information. This work lays a foundation for the construction of future advanced neuromorphic computing systems.

show abstract

Section: Resultsmentioning

confidence: 99%

Memristor‐Based Biologically Plausible Memory Based on Discrete and Continuous Attractor Networks for Neuromorphic Systems

Wang

Liu

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…As for ex situ training RNNs, Mahmoodi et al realized versatile stochastic dot product circuits based on a 20 × 20 passive stochastic memristive array. [ 142 ] The stochastic dot‐product operation could be used in the simulating annealing algorithms, which updated the neuron activations in a network with fixed weights to find the minimum energy (the lower energy meant a better optimized state), thus they demonstrated stochastic HNNs with a 64 × 64 array for the annealing algorithms. [ 143 ] With the help of the simulating annealing algorithms, the HNN successfully solved combinatory optimization problems, i.e., weighted maximum‐clique problems, weighted vertex cover problem, independent set problem, and graph partitioning optimization problem.…”

Section: Memristive Neural Networkmentioning

confidence: 99%

Artificial Neural Networks Based on Memristive Devices: From Device to System

Huang

Wang

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

Thanks to the advance in transistor scaling, computing capabilities of central processing units (CPUs) and graphics processing units (GPUs) have been increased greatly, resulting in the fast development of artificial neural networks (ANNs). ANNs predict the correlation between a set of input and output parameters by emulating the principles of biological neural networks. [1] Nowadays, ANNs have been applied to a broad spectrum, such as computer vision, natural language processing, autonomous driving, and decision making. Although ANNs are advantageous in some specific tasks, such as playing the game of Go, [2,3] the human brain outperforms ANNs in cognitive and classification tasks with fast speed and high power efficiency. The advantages of the human brain lie in the special architectures with high massive parallelism, spiking neurons, and enormous number of basic computing units of neurons (%10 11) and synapses (%10 15), whereas computers show high speed of basic operation and precision. [4] The massively parallel processing compensates the low speed of neurons and synapses, whereas spiking neurons compute more powerful than other types of neurons. [5] The parallel processing occurs based on variable synaptic weights, which is the basis of memory. Moreover, the same information is processed by many neurons and transmitted to a downstream neuron, which enhances the precision as well. Therefore, the human brain shows a salient feature, i.e., in-memory computing, suitable for high-performance applications with a large amount of data. In contrast, conventional digital computing systems face the Von Neumann bottleneck, because in the Von Neumann architecture, computing units and memory are separated, hindering the enhancement of ANNs. Thus, several novel architectures are promoted, such as brain-inspired computing (processing information based on spiking neurons, also called as spiking neural networks [SNNs]) and in-memory computing on chip, to overcome the Von Neumann bottleneck. [6] To implement the new architectures, artificial synapses and neurons are necessary. The massive parallelism of the brain takes advantage of a large number of neurons and synapses; however, the emulation of the functions of biological synapses and neurons requires tens of complement metal-oxide-semiconductor (CMOS) transistors. Therefore, it is urgent to realize synapses and neurons with a simplified method for the development of ANNs. [7] As synapses adjust their weights step-by-step repeatedly, synaptic devices need to exhibit analog switching behavior. [8] However, biological neurons show the analog membrane potential and generate the all-or-nothing rapidly, so both analog and digital devices are acceptable for the emulation of neurons. Many emerging devices have been proposed to develop ANNs based on hardware, e.g., memristive devices, phase change memories (PCMs), [9] magnetic random-access memories (MRAMs), [10] and ferroelectric field-effect transistors (FeFETs). [11] Each technology has its advantages and disadvantages. PCM and M...

show abstract

“…where W presents the weight to be stored, R p0 and R n0 are the corresponding memristance that is mapped into ''positive'' and ''negative'' values without tuning, respectively, R p and R n are the memristance updated by (11) and (12), respectively, and R min and R max are the minimum and maximum memristor resistances, respectively.…”

Section: A Proposed Recursive Circuit Schemementioning

confidence: 99%

Memristor-Based Analog Recursive Computation Circuit for Linear Programming Optimization

Shang

Adil

Madani

et al. 2020

IEEE J. Explor. Solid-State Comput. Devices Circuits

View full text Add to dashboard Cite

Linear programming optimization is central to engineering designs, logistics management, and decision-making in every sector of the economy. Traditional hardware using CPU and GPU platforms for this purpose is severely limited by the scaling of the transistor technology. In this article, we design an analog in-memory computation circuit for accelerating linear programming optimization problems. The scheme includes a memristive crossbar array and analog peripheral circuits that do not require DAC/ADC between each algorithm iteration. In addition, several key parameters related to nonideal device characteristics and interconnect parasitics are discussed for providing practical guidelines. Furthermore, three design schemes are proposed to alleviate the computation error caused by the interconnect resistance for a large-scale crossbar array implementation. Optimal design parameters are quantified under a given number of array size and memristive resistance. Finally, the proposed hardware accelerator and error mitigation techniques are applied to six real-world power system optimization problems. The results show that the average error of generator power and the overall cost is less than 3%. It is demonstrated that the proposed accelerator achieves area, delay, and energy consumption reductions of ∼151×, ∼33×, and ∼21×, respectively, compared with the CMOS digital circuits at the 16-nm technology node for a 1000 × 1000 array with 6-bit precision.

show abstract

Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization

Cited by 80 publications

References 61 publications

Memristor‐Based Biologically Plausible Memory Based on Discrete and Continuous Attractor Networks for Neuromorphic Systems

Memristor‐Based Biologically Plausible Memory Based on Discrete and Continuous Attractor Networks for Neuromorphic Systems

Artificial Neural Networks Based on Memristive Devices: From Device to System

Memristor-Based Analog Recursive Computation Circuit for Linear Programming Optimization

Contact Info

Product

Resources

About