Implementing regularly structured neural networks on the DREAM machine

Abstract

High-throughput implementations of neural network models are required to transfer the technology from small prototype research problems into large-scale "real-world" applications. The flexibility of these implementations in accommodating for modifications to the neural network computation and structure is of paramount importance. The performance of many implementation methods today is greatly dependent on the density and the interconnection structure of the neural network model being implemented. A principal contribution of this paper is to demonstrate an implementation method which exploits maximum amount of parallelism from neural computation, without enforcing stringent conditions on the neural network interconnection structure, to achieve this high implementation efficiency. We propose a new reconfigurable parallel processing architecture, the Dynamically Reconfigurable Extended Array Multiprocessor (DREAM) machine, and an associated mapping method for implementing neural networks with regular interconnection structures. Details of the system execution rate calculation as a function of the neural network structure are presented. Several example neural network structures are used to demonstrate the efficiency of our mapping method and the DREAM machine architecture on implementing diverse interconnection structures. We show that due to the reconfigurable nature of the DREAM machine, most of the available parallelism of neural networks can be efficiently exploited.

FPGA Implementation of a Pipelined On-Line Backpropagation

“…The second variation is based on the fact that the computations in a neural network are basically matrix products [16][17][18][19]. The advantage of this approach is the amount of data communicated between processors is moderate and evenly distributed, although in a multilayer perceptron the synaptic matrix is lower triangular.…”

An Optimal Implementation on FPGA of a Hopfield Neural Network

“…Examples can be found in [7][8][9][10][11][12][13][14][15][16][17]. In this paper, an implementation of HNN into an SRAM-based FPGA is shown.…”

“…From the simulation results presented in the previous section it can be clearly shown that a HNN of four nodes will only A comparison with previous work relating generalpurpose parallel machines shows the performance superiority of our implementation over known implementations on planar architectures. Most known implementations [11,13,14,16,17] require O(N) time complexity, whereas the proposed implementation requires O(log N) time complexity. Implementations on nonplanar architectures, such as hypercube, show a minor performance gain over our design at the cost of much more complex interconnection network [10,15].…”

Design and analysis of maximum Hopfield networks

“…Thus, the true advantage of using Hopfield-type neural networks to solve difficult optimization problems relates to speed considerations. Due to their inherently parallel strucManuscript received August 23, 1999 ture and simple computational requirements, neural network techniques are especially suitable for direct hardware implementation, using analog or digital integrated circuits [22], or parallel simulations [23]. Moreover, the Hopfield neural networks have very natural implementations in optics [24].…”