PERTURBATION TECHNIQUES FOR ON-CHIP LEARNING WITH ANALOGUE VLSI MLPs

Abstract

Microelectronic neural network technology has become sufficiently mature over the past few years that reliable performance can now be obtained from VLSI circuits under carefully controlled conditions (see Refs. 8 or 13 for example). The use of analogue VLSI allows low power, area efficient hardware realisations which can perform the computationally intensive feed-forward operation of neural networks at high speed, making real-time applications possible. In this paper we focus on important issues for the successful operation and implementation of on-chip learning with such analogue VLSI neural hardware, in particular the issue of weight precision. We first review several perturbation techniques which have been proposed to train multi-layer perceptron (MLP) networks. We then present a novel error criterion which performs well on benchmark problems and which allows simple integration of error measurement hardware for complete on-chip learning systems.

The Prospects for Analogue Neural VLSI

“…69 Another strategy, the one on which we concentrate here, is to make some alteration to the standard back-propagation algorithm which will simplify the hardware implementation but retain the essentials of the algorithm. 70,71 The examples we give here are: the virtual targets algorithm, which reduces the difference between the hidden and output layers, at the expense of introducing explicit targets for the hidden layer; and the weight perturbation algorithm, which avoids calculating error/weight gradients by measuring them instead.…”