Hardware implementation of stochastic-based Neural Networks

Abstract

In this work we review the basic principles of stochastic logic and its application to the hardware implementation of Neural Networks. In this paper we show the mathematical basis of stochastic-based neurons along with the specific circuits that are needed to implement the processing of each neuron. We also propose a new methodology to reproduce the non-linear activation function. The proposed methodology can be used to implement any kind of Neural Network

Probabilistic-based neural network implementation

“…A new coding can be done using numbers represented by the ratio of the switching activity of different bipolar coded stochastic signals. With this notation data/values are fully represented by two values (the numerator and the denominator) rather than with a single stochastic signal As happens with the traditional stochastic logic, binary numbers can be converted to stochastic signals using Random Number Generators (RNG) and comparators for the numerator and the denominator [3,4,23]. These blocks are defined as Binary to Pulse converters (B2P).…”

“…Otherwise, to convert a switching signal to a binary quantity a counter operating during a certain period of time is needed. With coupled counters and a memory register we can implement the Pulse to Binary (P2B) converter [3,23]. Therefore, combining two P2B blocks we can translate two stochastic signals (p,q) to binary numbers (P,Q).…”

“…In consequence the implementation of large neural networks in hardware only is possible if the associated circuitry to the neural multipliers is reduced significantly. A feasible way to implement neural networks in hardware is the use of stochastic computing concepts [3][4][5][6][7][8][9]. With this computing scheme, probabilistic laws are applied to logic cells and stochastic pulse streams are used as analog variables [10][11][12].…”

“…As an example, with the stochastic computing framework (unipolar codification) multiplication is performed using a single AND gate and the addition between two signals can be approximated using an OR gate. All this has led an intensive work devoted to implement Neural Networks by using stochastic computing concepts [3][4][5][6][7][8][9], in order to open the possibility to implement large neural systems in a single chip.…”

On the Functions Realized by Stochastic Computing Circuits

“…Since then, it has found its major applications in control systems [29] and artificial neural networks [7] [26]. More recently, new applications have appeared that involve probabilistic or error-tolerance issues for which SC is well suited, such as image processing [2] [17], simulation of probabilistic systems [9] [21], data recognition and mining [11], and decoders for channel codes ranging from LDPC to polar codes [13] [28].…”