Learning with analogue VLSP MLPs

Abstract

Much work has been undertaken to demonstrate the advantages of analogue V U 1 for implementing neural architectures. This paper attempts to address the issues concerning 'in-situ' learning with analogue V U 1 multi-layer perceptron ( M U ) networks. In particular, we propose that 'chip-in-the-loop' learning is, at the very least, necessary to overcome typical analogue process variations and we argue that MLPs containing analogue circuits with 8 bit precision can be successfully trained provided they have digital representations of the weights of at least 12 bits. We demonstrate that weight perturbation, with careful choice of the perturbation size, gives improved results over backpropagation, at the cost of increased training time. Indeed, we go on to show why weight perturbation is possibly the only sensible way to implement MLP 'on-chip' learning.We have designed a set of analogue V U I chips specifically to see ifour theoretical results on learning work in practice.Although these chips are experimental, it is our intention to use them to solve 'real world' problems which have relatively low input dimensionality, such as the task of speaker identijication.

Non-overlapped implantation (NOI) MOSFET synapse and its implementation on supervised neural network

“…Fig. 6 depicts the perceptron architecture that has been implemented [11,16]. In the above mentioned supervised ANN algorithm, the weight Wt ij , weight update ΔWt ij , learning rate η, input P i , target T j and target error E j of each neuron j are respectively defined in the following,…”

“…In addition, the advantages of hardware realizations include high computation speed and relative ease of integration with analog interface. There are many varieties of structures and computational methods containing digital and analog forms in hardware neural implementation in past studies [7][8][9][10][11][12][13][14]. Among different concepts, the NVM device is considered the most promising for its analog memory nature and small chip area.…”

Characterization of Analog Local Cluster Neural Network Hardware for Control

“…For , we choose the logistic sigmoid function (7) where is an affine transform of (8) We can rewrite the sigmoid function as …”

Local Cluster Neural Network On-chip Training

“…Calculation of the output sigmoids as equation (7). The constant b allows shifting the window with respect to the function.…”

“…There are three schemes for training neural net hardware [7]. Off-chip training computes the network weights in separate computer simulation and then downloads the weight onto the chip.…”