Continuous-valued probabilistic behavior in a VLSI generative model

Abstract

This paper presents the VLSI implementation of the continuous restricted Boltzmann machine (CRBM), a probabilistic generative model that is able to model continuous-valued data with a simple and hardware-amenable training algorithm. The full CRBM system consists of stochastic neurons whose continuous-valued probabilistic behavior is mediated by injected noise. Integrating on-chip training circuits, the full CRBM system provides a platform for exploring computation with continuous-valued probabilistic behavior in VLSI. The VLSI CRBM's ability both to model and to regenerate continuous-valued data distributions is examined and limitations on its performance are highlighted and discussed.

A building block for hardware belief networks

“…… it simulates nature in this sense : that C goes from some .. initial state .. to some final state with the same probability that N goes from the corresponding initial state to the corresponding final state . … If you repeat the same experiment in the computer a large number of times … it will give the frequency of a given final state proportional to the number of times , with approximately the same rate … as it happens in nature .” The possibility of probabilistic computing machines has also been addressed by more recent authors 16 17 18 19 20 21 22 . The primary purpose of this paper is to introduce the concept of a ‘transynapse’, a device that can be interconnected in large numbers to build probabilistic computers ( Fig.…”