Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning

Bojnordi, Mahdi Nazm; Ïpek, Engin

doi:10.1109/e3s.2017.8246178

Cited by 17 publications

(19 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the area required to implement the RC circuit with 100 K Ω resistor and 20f F capacitor is approximately three times larger than that of the MRAM-based neuron [Scott 1998;Stengel and Spaldin 2006]. Thus, the proposed MRAM-based activation function can achieve approximately 20× and 300× area reduction compared to the CMOS-based stochastic neurons proposed in [Ardakani et al 2017] and [Bojnordi and Ipek 2016], respectively. The area of the MRAM-based neuron, which is utilized as the baseline for the area comparisons, is approximately equal to 32λ × 32λ, that is obtained by the layout design, in which λ is a technology-dependent parameter.…”

Section: Discussionmentioning

confidence: 98%

“…On the other hand, emerging technologies such as resistive RAM (RRAM) and phase change memory (PCM) have been recently utilized within the crossbar arrays to implement matrix multiplication within RBMs [Bojnordi and Ipek 2016;Eryilmaz et al 2016;Sheri et al 2015]. In particular, [Bojnordi and Ipek 2016] has achieved 100× and 10× improvement in terms of operation speed and energy consumption, respectively, compared to single-threaded cores by using RRAM devices as weighted connections.…”

Section: Discussionmentioning

confidence: 99%

“…Previous work on RBM hardware implementations use conventional VLSI design techniques [Yuan and Parhi 2017], FPGA approaches [Kim et al 2010;Ly and Chow 2010], and stochastic CMOS methods [Ardakani et al 2017]. Moreover, emerging technologies such as resistive RAM (RRAM) [Bojnordi and Ipek 2016;Sheri et al 2015] and phase change memory (PCM) [Eryilmaz et al 2016] had been utilized as weighted connections within the DBN architecture to interconnect its various building blocks. The previous hybrid Memristor/CMOS designs attempt to realize an intrinsic implementation of the weighted connections.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons

Zand

Datta

DeMara

2019

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

Magnetoresistive random access memory (MRAM) technologies with thermally unstable nanomagnets are leveraged to develop an intrinsic stochastic neuron as a building block for restricted Boltzmann machines (RBMs) to form deep belief networks (DBNs). The embedded MRAM-based neuron is modeled using precise physics equations. The simulation results exhibit the desired sigmoidal relation between the input voltages and probability of the output state. A probabilistic inference network simulator (PIN-Sim) is developed to realize a circuit-level model of an RBM utilizing resistive crossbar arrays along with differential amplifiers to implement the positive and negative weight values. The PIN-Sim is composed of five main blocks to train a DBN, evaluate its accuracy, and measure its power consumption. The MNIST dataset is leveraged to investigate the energy and accuracy tradeoffs of seven distinct network topologies in SPICE using the 14nm HP-FinFET technology library with the nominal voltage of 0.8V, in which an MRAM-based neuron is used as the activation function. The software and hardware level simulations indicate that a 784 × 200 × 10 topology can achieve less than 5% error rates with ∼ 400p J energy consumption. The error rates can be reduced to 2.5% by using a 784 × 500 × 500 × 500 × 10 DBN at the cost of ∼ 10× higher energy consumption and significant area overhead. Finally, the effects of specific hardware-level parameters on power dissipation and accuracy tradeoffs are identified via the developed PIN-Sim framework.Additional Key Words and Phrases: Deep belief network (DBN), restricted Boltzmann machine (RBM), magnetoresistive random access memory (MRAM), stochastic binary neuron, resistive crossbar array

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons

Zand

Datta

DeMara

2019

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…Hamdioui et al [57] introduced CIM, a memristor based computation in memory architecture. Bojnordi and Ipek [58] introduced the Memristive Boltzmann machine, a hardware accelerator for combinatorial optimization and deep learning. Imani et al [59] introduced APIM, an approximate processing in-memory architecture which exploits the analog characteristics of resistive memory to support addition and multiplication inside the crossbar.…”

Section: Processing-in-resistive Memorymentioning

confidence: 99%

POSTER: BioSEAL: In-Memory Biological Sequence Alignment Accelerator for Large-Scale Genomic Data

Kaplan

Yavits

Ginosar

2019

2019 28th International Conference on Parallel Architectures and Compilation Techniques (PACT)

View full text Add to dashboard Cite

Genome sequences contain hundreds of millions of DNA base pairs. Finding the degree of similarity between two genomes requires executing a compute-intensive dynamic programming algorithm, such as Smith-Waterman. Traditional von Neumann architectures have limited parallelism and cannot provide an efficient solution for large-scale genomic data. Approximate heuristic methods (e.g. BLAST) are commonly used. However, they are suboptimal and still compute-intensive.In this work, we present BioSEAL, a Biological SEquence ALignment accelerator. BioSEAL is a massively parallel non-von Neumann processing-inmemory architecture for large-scale DNA and protein sequence alignment. BioSEAL is based on resistive content addressable memory, capable of energyefficient and high-performance associative processing.We present an associative processing algorithm for entire database sequence alignment on BioSEAL and compare its performance and power consumption with state-of-art solutions. We show that BioSEAL can achieve up to 57× speedup and 156× better energy efficiency, compared with existing solutions for genome sequence alignment and protein sequence database search.

show abstract

“…Most of the previous DBN research has focused on software implementations, which provides flexibility, but requires significant execution time and energy due to large matrix multiplications that are relatively inefficient when implemented on standard Von-Neumann architectures. Previous hardware-based implementation of RBM have sought to overcome software limitations by using FPGAs [11], [12], stochastic CMOS [13], and hybrid memristor-CMOS designs [14]. Recently, Zand et al [15] utilized a spintronic device that leverages intrinsic thermal noise within low energy barrier nanomagnets to provide a natural building block for RBMs.…”

Section: Introductionmentioning

confidence: 99%

SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks

Zand

DeMara

2018

2018 IEEE International Conference on Rebooting Computing (ICRC)

View full text Add to dashboard Cite

In this paper, a spintronic neuromorphic reconfigurable Array (SNRA) is developed to fuse together power-efficient probabilistic and in-field programmable deterministic computing during both training and evaluation phases of restricted Boltzmann machines (RBMs). First, probabilistic spin logic devices are used to develop an RBM realization which is adapted to construct deep belief networks (DBNs) having one to three hidden layers of size 10 to 800 neurons each. Second, we design a hardware implementation for the contrastive divergence (CD) algorithm using a four-state finite state machine capable of unsupervised training in N+3 clocks where N denotes the number of neurons in each RBM. The functionality of our proposed CD hardware implementation is validated using ModelSim simulations. We synthesize the developed Verilog HDL implementation of our proposed test/train control circuitry for various DBN topologies where the maximal RBM dimensions yield resource utilization ranging from 51 to 2,421 lookup tables (LUTs). Next, we leverage spin Hall effect (SHE)-magnetic tunnel junction (MTJ) based non-volatile LUTs circuits as an alternative for static random access memory (SRAM)-based LUTs storing the deterministic logic configuration to form a reconfigurable fabric. Finally, we compare the performance of our proposed SNRA with SRAMbased configurable fabrics focusing on the area and power consumption induced by the LUTs used to implement both CD and evaluation modes. The results obtained indicate more than 80% reduction in combined dynamic and static power dissipation, while achieving at least 50% reduction in device count.

show abstract

Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning

Cited by 17 publications

References 1 publication

Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons

Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons

POSTER: BioSEAL: In-Memory Biological Sequence Alignment Accelerator for Large-Scale Genomic Data

SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks

Contact Info

Product

Resources

About