Multilayer Perceptron Training Accelerator Using Systolic Array

Abstract

Takeshi SENOO †a) , Student Member, Akira JINGUJI †b) , Member, Ryosuke KURAMOCHI †c) , Nonmember, and Hiroki NAKAHARA †d) , Member SUMMARY Multilayer perceptron (MLP) is a basic neural network model that is used in practical industrial applications, such as network intrusion detection (NID) systems. It is also used as a building block in newer models, such as gMLP. Currently, there is a demand for fast training in NID and other areas. However, in training with numerous GPUs, the problems of power consumption and long training times arise. Many of the latest deep neural network (DNN) models and MLPs are trained using a backpropagation algorithm which transmits an error gradient from the output layer to the input layer such that in the sequential computation, the next input cannot be processed until the weights of all layers are updated from the last layer. This is known as backward locking. In this study, a weight parameter update mechanism is proposed with time delays that can accommodate the weight update delay to allow simultaneous forward and backward computation. To this end, a one-dimensional systolic array structure was designed on a Xilinx U50 Alveo FPGA card in which each layer of the MLP is assigned to a processing element (PE). The time-delay backpropagation algorithm executes all layers in parallel, and transfers data between layers in a pipeline. Compared to the Intel Core i9 CPU and NVIDIA RTX 3090 GPU, it is 3 times faster than the CPU and 2.5 times faster than the GPU. The processing speed per power consumption is 11.5 times better than that of the CPU and 21.4 times better than that of the GPU. From these results, it is concluded that a training accelerator on an FPGA can achieve high speed and energy efficiency.

Efficient On-Chip Learning of Multi-Layer Perceptron Based on Neuron Multiplexing Method

“…The number in the bracket in the column of "Type" indicates the NCU number used in the proposed method. Compared with the method in [27,30,38], our method may be inferior in terms of calculation speed. However, the proposed method has a significant advantage in terms of resource consumption, which becomes more pronounced as the number of NCUs decreases.…”

“…Further, the proposed method is compared with the method in [27,30,38]. The comparison results are shown in Tables 7-9.…”

“…The pipelined structure, proposed in [29], facilitated online learning of the neural network on FPGA by executing inference and learning in a pipelined manner to accelerate the training process. In [30], a time-delay backpropagation algorithm was proposed, which supported simultaneous forward and backward computation and avoided the problem of backward locking. Despite the advantage of fast calculation, the pipelining strategy was resource intensive.…”