An analog neural network chip with random weight change learning algorithm

Abstract

Although researchers have been engaged in fabrication of neural network hardware, only a few networks implemented with a learning algorithm have been reported. A learning algorithm is required to be implemented on a VLSI chip because off-chip learning with a digital computer consumes too much time to be applied to many practical problems. The main obstacle to implement a learning algorithm is the complexity of the proposed algorithms. Algorithms like Back Propagation include complex multiplication, summation and derivatives, which are very difficult to implement with VLSI circuits. The authors propose a new learning algorithm, which is suitable for analog implementation , and implemented it on a 2.2 mm x 2.2 mm neural network chip with 100 weights, using the standard 2.0 pm MOSIS process. The chips have successfully learned the XOR Gate problem.

A CMOS feedforward neural-network chip with on-chip parallel learning for oscillation cancellation

“…Learning rules like serial-weight-perturbation [3] (or Madaline Rule III) and the chain perturbation rule [4] are very tolerant of the analog circuit nonidealities, but they are either serial or partially parallel computation algorithms, thus are often too slow for real-time control. In this research, we use the RWC algorithm [1], which is a fully parallel rule that is insensitive to circuit nonidealities and can be used in direct feedback control. The RWC algorithm is defined as follows.…”

“…Categorized by storage types, there are five kinds of synapse circuits: capacitor only [1], [7]- [11], capacitor with refreshment [12]- [14], capacitor with EEPROM [4], digital [15], [16], and mixed D/A [17] circuits.…”

“…Hardware-friendly algorithms are essential to ensure the functionality and cost effectiveness of the hardware implementation. In this research, a hardware-friendly algorithm, called random-weight-change (RWC) algorithm [1], is implemented on CMOS processes. The RWC algorithm is a fully parallel rule that is insensitive to circuit nonidealities.…”

A fast on-line neural-network training algorithm for a rectifier regulator

“…A new ANN training algorithm called random weight change (RWC) has been developed as a variation of a previously proposed method of ANN training based on random search for a minimum on the error surface [9]. As opposed to the deterministic methods of weight training like backpropagation, the RWC algorithm is a statistical or probabilistic method.…”

“…To implement backpropagation training in hardware requires high-precision multiplication [9], and this limits the size and/or speed of the hardware that can be fabricated. For applications with hundreds of weights and weight update times in the microseconds, the RWC algorithm hardware is, therefore, superior in the cost and complexity of the hardware required.…”