Analog VLSI neural network with digital perturbative learning

Abstract

Abstract-Two feed-forward neural-network hardware implementations are presented. The first uses analog synapses and neurons with a digital serial weight bus. The chip is trained in loop with the computer performing control and weight updates. By training with the chip in the loop, it is possible to learn around circuit offsets. The second neural network also uses a computer for the global control operations, but all of the local operations are performed on chip. The weights are implemented digitally, and counters are used to adjust them. A parallel perturbative weight update algorithm is used. The chip uses multiple, locally generated, pseudorandom bit streams to perturb all of the weights in parallel. If the perturbation causes the error function to decrease, the weight change is kept; otherwise, it is discarded. Test results from a very large scale integration (VLSI) prototype are shown of both networks successfully learning digital functions such as AND and XOR.Index Terms-Analog very large scale integration (VLSI), chip-in-loop training algorithm, learning, neural chips, neural network, neuromorphic, perturbation techniques, VLSI feed-forward neural network.

The potential of fuzzy neural networks in the realization of approximate reasoning engines

“…In general, development of these systems seems to be focused towards digital systems, although there has been some work done in the analog domain [10,42]. Current neural network hardware technological advancements were more closely examined, and it was found that while there was a broad range of implementation techniques, the majority of efforts could be classified as an analog [1,9,27,2], digital [11,37,44,48,49], or hybrid (mixed-signal) [30,18,19] solution. Analog systems are preferred for large productions, very low power, very high sample rate or bandwidth, and small size.…”

“…However, when considering a hardware implementation, it would be beneficial to avoid a backpropagating method, which requires difficult circuit implementations and costly gradient calculations. Instead, methods of simultaneous perturbation (SP) are suitable here, having already achieved success in hardware implementations [18,24]. The advantage of these methods is simplicity, only needing values of the performance index for making weight adjustments.…”

Efficient hardware implementation of threshold neural networks

“…Hardware implementation of neural networks has been used in applications such as pattern recognition [1], olfaction system [2], test of analog circuits [3], range-finding [4] and smart sensing [5].…”

A Reconfigurable Gaussian/Triangular Basis Functions Computation Circuit

“…Depending on the intended application of the neural network [1] and [3], the activation function has a linear, a piece-wise linear or a nonlinear relationship between its input and output. The Gaussian (Radial Basis Function -RBF) and Triangular (Triangular Basis Function -TBF) functions are widely used in different types of Neural Networks, such as the activation function in radial basis function networks [6].…”