A Software/Hardware Co-design Local Irregular Sparsity Method for Accelerating CNNs on FPGA

Shang, Jiangwei; Zhang, Zhan; Li, Chuanyou; Zhang, Kun; Qian, Lei; Liu, Hongwei

doi:10.1145/3547276.3548521

Cited by 1 publication

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we extend our previous work 21 from both the algorithm and hardware accelerator. For the algorithm, we extend the sparsifying method to prune pointwise convolutional layers and show the index representations of sparse convolutional weights.…”

Section: Introductionmentioning

confidence: 80%

An algorithm/hardware co‐optimized method to accelerate CNNs with compressed convolutional weights on FPGA

Shang,

Zhang,

Zhang

et al. 2024

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SummaryConvolutional neural networks (CNNs) have shown remarkable advantages in a wide range of domains at the expense of huge parameters and computations. Modern CNNs still tend to be more complex and larger to achieve better inference accuracy. However, the complex and large structures of CNNs could slow down the inference speed. Recently, Compressing the convolutional weights to be sparse by pruning the unimportant parameters has been demonstrated as an efficient way to reduce the computations of CNNs. On the other hand, field‐programmable gate arrays (FPGAs) have been a popular hardware platform to accelerate CNN inference. In this paper, we propose an algorithm/hardware co‐optimized method for accelerating CNN inference on FPGAs. For the algorithm, we take advantage of unstructured and structured parameter sparsifying methods to achieve high sparsity and keep the regularity of convolutional weights. Correspondingly, hardware‐friendly index representations of sparse convolutional weights are proposed. For the hardware architecture, we propose row‐wise input‐stationary dataflow, which is tightly coupled with the algorithm. A row‐wise computing engine (RConv Engine) is proposed, which is based on the dataflow. Inside the RConv Engine, the scalar‐vector structure is applied to implement the basic processing elements (PEs). To flexibly calculate the feature map with various sizes, the PEs are organized in a 2D structure with two work modes. The experimental results demonstrate that our co‐optimized method implements high sparsity of convolutional weights, and the computing engine achieves high computation efficiency. Compared with other accelerators, our co‐optimized method implements a 10.9 speedup on FPS at most with the highest sparsity of convolutional weights and negligible accuracy loss.

show abstract

Section: Introductionmentioning

confidence: 80%