Array Aware Training/Pruning: Methods for Efficient Forward Propagation on Array-based Neural Network Accelerators

Abstract

Due to the increase in the use of large-sized Deep Neural Networks (DNNs) over the years, specialized hardware accelerators such as Tensor Processing Unit and Eyeriss have been developed to accelerate the forward pass of the network. The essential component of these devices is an array processor which is composed of multiple individual compute units for efficiently executing Multiplication and Accumulation (MAC) operation. As the size of this array limits the amount of DNN processing of a single layer, the computation is performed in several batches serially leading to extra compute cycles along both the axes. In practice, due to the mismatch between matrix and array sizes, the computation does not map on the array exactly. In this work, we address the issue of minimizing processing cycles on the array by adjusting the DNN model parameters by using a structured hardware array dependent optimization. We introduce two techniques in this paper: Array Aware Training (AAT) for efficient training and Array Aware Pruning (AAP) for efficient inference. Weight pruning is an approach to remove redundant parameters in the network to decrease the size of the network. The key idea behind pruning in this paper is to adjust the model parameters (the weight matrix) so that the array is fully utilized in each computation batch. Our goal is to compress the model based on the size of the array so as to reduce the number of computation cycles. We observe that both the proposed techniques results into similar accuracy as the original network while saving a significant number of processing cycles (75%).

Analisis Sentimen Twitter untuk Menilai Opini Terhadap Perusahaan Publik Menggunakan Algoritma Deep Neural Network

“…Salah satu penerapan dari neural network adalah Deep Neural Network atau DNN. Deep Neural Network adalah neural network yang tersusun dari layer yang jumlahnya lebih dari satu [11]. Penelitian yang menggunakan DNN dilakukan oleh [12] untuk menganalisa sentimen pada Twitter berbahasa Indonesia mengenai institusi pemerintahan dan tokoh pemerintahan.…”

“…Hasil perhitungan ini kemudian dihitung lagi dengan menggunakan fungsi aktivasi yang merupakan keluaran dari layer tersebut. Persamaan 1 merupakan perhitungan pada setiap layer[11].…”

Efficient Design Space Exploration for Sparse Mixed Precision Neural Architectures

“…Neural Network model optimization techniques such as efficient network design, pruning, and quantization are essential for efficient inference on hardware in real-time. Pruning [4,5,11,12] is a method to reduce the size and computational complexity of a network as it removes redundant connections/neurons which do not significantly contribute to the model accuracy. Weight/Connection-wise pruning is irregular in nature and hence introduces non-uniform sparsity in the weight matrices.…”