Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning

Abstract

The Boltzmann machine is a massively parallel computational model capable of solving a broad class of combinatorial optimization problems. In recent years, it has been successfully applied to training deep machine learning models on massive datasets. High performance implementations of the Boltzmann machine using GPUs, MPI-based HPC clusters, and FPGAs have been proposed in the literature. Regrettably, the required all-to-all communication among the processing units limits the performance of these efforts.This paper examines a new class of hardware accelerators for large-scale combinatorial optimization and deep learning based on memristive Boltzmann machines. A massively parallel, memory-centric hardware accelerator is proposed based on recently developed resistive RAM (RRAM) technology. The proposed accelerator exploits the electrical properties of RRAM to realize in situ, fine-grained parallel computation within memory arrays, thereby eliminating the need for exchanging data between the memory cells and the computational units. Two classical optimization problems, graph partitioning and boolean satisfiability, and a deep belief network application are mapped onto the proposed hardware. As compared to a multicore system, the proposed accelerator achieves 57× higher performance and 25× lower energy with virtually no loss in the quality of the solution to the optimization problems. The memristive accelerator is also compared against an RRAM based processing-in-memory (PIM) system, with respective performance and energy improvements of 6.89× and 5.2×.978-1-4673-9211-2/16/$31.00 c 2016 IEEE 2 2

Memristive Devices and Networks for Brain‐Inspired Computing

“…Based on the neuromorphic devices and arrays shown above, neuromorphic computing systems can be further constructed. Since the VMM operation can be accelerated in memristor crossbar thanks to the parallel, in‐memory, and analog characteristics of memristors, a large variety of neural networks can be accelerated, and neuromorphic systems capable of encoding and processing spatiotemporal information might also be built by exploiting device dynamics such as short‐term plasticity . Figure summarizes typical network‐level demonstrations to date that have been achieved experimentally, grouped according to the type of devices and algorithms used.…”

“…Due to the limitations in device performance, memristive hardware is commonly considered more suitable for circumstances where the algorithm itself can tolerate device variations or relative lower precisions . In stark contrast to ANN, there are still many tasks that involve high precision computing, such as numerical simulations.…”

MB-CNN: Memristive Binary Convolutional Neural Networks for Embedded Mobile Devices

“…Resistive technologies have shown excellent characteristics for the future memory systems capable of storing large amounts of data and performing in-memory computation [31][32][33]. Resistive RAM (RRAM) is one of the most promising memristive devices currently under commercial development that exhibits excellent scalability, high-speed switching, a high dynamic resistance range that permits multi-level cells (MLC), and low power consumption [34].…”

Recent trends in neuromorphic engineering

“…k-means, k-nearest neighbors, naive bayes, support vector machines, linear regression, classification trees and deep neural networks. Bojnordi et al [28] develop a memristive Boltzmann machine for large scale combinatorial optimization and deep learning. They demonstrate their accelerator on the graph partitioning and boolean satisfiability problems, and obtain 57× higher performance and 25× lower energy.…”