On-Chip Learning With Analogue Vlsi Neural Networks

Abstract

Results from simulations of weight perturbation as an on-chip learning scheme for analogue VLSI neural networks are presented. The limitations of analogue hardware are modelled as realistically as possible. Thus synaptic weight precision is defined according to the smallest change in the weight setting voltage which gives a measurable change at the output of the corresponding neuron. Tests are carried out on a hard classification problem constructed from mobile robot navigation data. The simulations show that the degradation in classification performance on a 500-pattern test set caused by the introduction of realistic hardware constraints is acceptable: with 8-bit weights, updated probabilistically and with a simplified output error criterion, the error rate increases by no more than 7% when compared with weight perturbation implemented with full 32-bit precision.

Neural Networks in Analog Hardware — Design and Implementation Issues

“…Furthermore, various researchers have reported problems related to the precision available on the chip in the case of training algorithms such as backpropagation. 85,89,178,200 The off-chip learning performs all computation off the chip. Once the solution weight state has been found, the weights are downloaded to the chip.…”

“…153 Due to the fact that standard error backpropagation usually needs over 12 bits of resolution, 85,89,178,200,201 the calculation of the gradient of the error with respect to the weights is usually calculated on a host computer with the errors provided by the actual outputs of the chip in a chip-in-the-loop manner. 95,121 The backpropagation can be used with on-line and batch weight updating.…”

Nonlinear backpropagation: doing backpropagation without derivatives of the activation function

“…Also, NLBP seems superior to normal backpropagation for large learning rates which is important to hardware implementations (cf. [8]). In Fig.…”

Mixed analog/digital matrix-vector multiplier for neural network synapses

“…Though digital storage has a severe area penalty and floating gate devices are tedious to program, successful system implementations of both types have been reported in the literature [2,3,4], primarily recall mode systems, however. For analog systems with on-chip learning simple capacitive storage [5] seems to be the more natural choice because the value stored in such a memory is easy to adjust and is not subject to discretizing effects which usually degrades learning [6]. Weight deterioration in such systems is a serious problem, though.…”