A Parametrizable High-Level Synthesis Library for Accelerating Neural Networks on FPGAs

Abstract

In recent years, Convolutional Neural Network CNN have been incorporated in a large number of applications, including multimedia retrieval and image classification. However, CNN based algorithms are computationally and resource intensive and therefore difficult to be used in embedded systems. FPGA based accelerators are becoming more and more popular in research and industry due to their flexibility and energy efficiency. However, the available resources and the size of the on-chip memory can limit the performance of the FPGA accelerator for CNN. This work proposes an High-Level Synthesis HLS library for CNN algorithms. It contains seven different streaming-capable CNN (plus two conversion) functions for creating large neural networks with deep pipelines. The different functions have many parameter settings (e.g. for resolution, feature maps, data types, kernel size, parallelilization, accuracy, etc.), which also enable compile-time optimizations. Our functions are integrated into the HiFlipVX library, which is an open source HLS FPGA library for image processing and object detection. This offers the possibility to implement different types of computer vision applications with one library. Due to the various configuration and parallelization possibilities of the library functions, it is possible to implement a high-performance, scalable and resource-efficient system, as our evaluation of the MobileNets algorithm shows.

A Cross-Platform OpenVX Library for FPGA Accelerators

“…HiFlipVX is an open source HLS FPGA library for image processing applications [2]. It has been extended for object recognition, which involves feature detection [22] and neural networks [23]. HiFlipVX is a C++ based library containing 53 functions, which are highly optimized and parametrizable using templates.…”

“…The analysis functions usually have to perform a complete analysis of the input image, such as creating a histogram. Other functions that operate, for example, on feature vectors [22] or on tensors [23] can be classified into the mentioned categories.…”

A High-Level Synthesis Library for Synthesizing Efficient and Functional-Safe CNN Dataflow Accelerators

“…hls4ml [7] allows the translation of trained Python models to HLS-ready C++ models. L. Kalms et al [8] proposed a HLS library that explores multiple parallelization schemes to improve efficiency and performance. Similarly, fpgaConvNet [9], Caffeine [10], and FlexCNN [11] use HLS to design CNN accelerators offering also various performance tuning strategies.…”

A Pruning Method Based on Feature Map Similarity Score

“…This puts forward higher requirements for model deployment. In some scenarios wherein resources, power consumption, and computing power are limited, it is difficult to deploy algorithms into applications [5,6].…”