Contrastive divergence for memristor-based restricted Boltzmann machine

Sheri, Ahmad Muqeem; Rafique, Aasim; Pedrycz, Witold; Jeon, Moongu

doi:10.1016/j.engappai.2014.09.013

Cited by 40 publications

(27 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The key computations for restricted Boltzmann machines, including the contrastive divergence most frequently used for training, are strongly dominated by fetching weight values and computing vector operations with them, similar in that sense to other machine learning systems 39 described above. Hence, computations in memristor arrays 40 can in principle provide significant accelerations in this computing application. In particular, ref.…”

Section: Nature Electronicsmentioning

confidence: 99%

The future of electronics based on memristive systems

2018

View full text Add to dashboard Cite

Section: Nature Electronicsmentioning

confidence: 99%

The future of electronics based on memristive systems

2018

View full text Add to dashboard Cite

“…al. propose a spiking neural network based on memristive synapses that implements a single step contrastive divergence algorithm for machine learning [43]. Each synapse comprises two memristors representing limited-precision positive weights.…”

Section: Neuromorphic Networkmentioning

confidence: 99%

Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning

Bojnordi

Ïpek

2016

2016 IEEE International Symposium on High Performance Computer Architecture (HPCA)

181

View full text Add to dashboard Cite

The Boltzmann machine is a massively parallel computational model capable of solving a broad class of combinatorial optimization problems. In recent years, it has been successfully applied to training deep machine learning models on massive datasets. High performance implementations of the Boltzmann machine using GPUs, MPI-based HPC clusters, and FPGAs have been proposed in the literature. Regrettably, the required all-to-all communication among the processing units limits the performance of these efforts.This paper examines a new class of hardware accelerators for large-scale combinatorial optimization and deep learning based on memristive Boltzmann machines. A massively parallel, memory-centric hardware accelerator is proposed based on recently developed resistive RAM (RRAM) technology. The proposed accelerator exploits the electrical properties of RRAM to realize in situ, fine-grained parallel computation within memory arrays, thereby eliminating the need for exchanging data between the memory cells and the computational units. Two classical optimization problems, graph partitioning and boolean satisfiability, and a deep belief network application are mapped onto the proposed hardware. As compared to a multicore system, the proposed accelerator achieves 57× higher performance and 25× lower energy with virtually no loss in the quality of the solution to the optimization problems. The memristive accelerator is also compared against an RRAM based processing-in-memory (PIM) system, with respective performance and energy improvements of 6.89× and 5.2×.978-1-4673-9211-2/16/$31.00 c 2016 IEEE 2 2

show abstract

“…The output voltage can be seen as a division between these two resistances as in Eq. (9). This voltage will be used to program the crossbar based on a set of predetermined weights.…”

Section: Ex-situ Training Processmentioning

confidence: 99%

“…This paper also presents the first memristor circuit for processing an RBM neural network. An RBM based approach was described in [9], although a detailed circuit for carrying out the operation not covered. This paper is also the first to describe how a memristor based convolutional neural network can be developed.…”

Section: Introductionmentioning

confidence: 99%

Memristor based neuromorphic circuit for ex-situ training of multi-layer neural network algorithms

Yakopcic

Hasan

Taha

2015

2015 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

This paper describes a novel memristor-based neuromorphic circuit that can be used for ex-situ training of multi-layer neural network algorithms. The presented ex-situ programming technique can be used to map many key neural algorithms directly onto the grid of resistances in a memristor crossbar. This is possible because the proposed circuit is capable of calculating a true dot product. Existing circuits provide an approximated dot product based on a summation of conductance values, which is inaccurate due to the parallel structure of the crossbar. To show the effectiveness and versatility of this circuit, three different powerful neural networks were simulated. These include a Restricted Boltzmann Machine (RBM) for character recognition, a Convolutional Neural Network (CNN) also for character recognition, and a Multilayer Perceptron (MLP) trained to perform Sobel edge detection. Finally, an architecture analysis was performedshowing that neuromorphic processors based on these memristor crossbar circuits can be up to 5 orders of magnitude more power efficient than a RISC processor.

show abstract

Contrastive divergence for memristor-based restricted Boltzmann machine

Cited by 40 publications

References 24 publications

The future of electronics based on memristive systems

The future of electronics based on memristive systems

Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning

Memristor based neuromorphic circuit for ex-situ training of multi-layer neural network algorithms

Contact Info

Product

Resources

About