On the RTL Implementation of FINN Matrix Vector Unit

Abstract

FPGA-based accelerators are becoming increasingly popular for deep neural network inference due to their ability to scale performance with increasing degree of specialization with dataflow architectures or custom data type precision. In order to reduce the barrier for software engineers and data scientists to adopt FPGAs, C++- and OpenCL-based design entries with high-level synthesis (HLS) have been introduced. They provide higher abstraction compared to register-transfer level (RTL)-based design. HLS offers faster development time, better maintainability and more flexibility in code exploration, when evaluating several options for multi-dimension tensors, convolutional layers or different degrees of parallelism. For this reason, HLS has been adopted by DNN accelerator generation frameworks such as FINN and hls4ml. In this paper, we present an alternative backend library for FINN, leveraging RTL. We investigate and evaluate, across a spectrum of design dimensions, the pros and cons of an RTL-based implementation versus the original HLS variant. We show that for smaller design parameters, RTL produces significantly smaller circuits as compared to HLS. For larger circuits, however, the look-up table (LUT) count of RTL-based design is slightly higher, up to around \(15\% \) . On the other hand, HLS consistently requires more flip-flops (FFs) (with an orders-of-magnitude difference for smaller designs) and block RAMs (BRAMs) (2 × more). This also impacts the critical path delay, with RTL producing significantly faster circuits, up to around \(80\% \) . Furthermore, RTL also benefits from at-least a 10 × reduction in synthesis time. Finally, the results were validated in practice using two real-world use cases, one of a multi-layer perceptron (MLP) used in network intrusion detection and the other a convolution network called ResNet used in image recognition. Overall, since HLS frameworks code-generate the hardware design, the benefits of the ease in the design entry is less important. As such, the gained benefits in synthesis time together with some design-dependent resource benefits, make the RTL abstraction an attractive alternative.

A critical review on the state-of-the-art and future prospects of machine learning for Earth observation operations

“…or third-party libraries such as Larq ("Larq | Binarized Neural Network development," 2022.) and FINN (Alam et al, 2022) respectively.…”

A Configurable Mixed-Precision Convolution Processing Unit Generator in Chisel

“…Such an approach is not completely novel [6,7]. However, previous work mostly focused on using high-level synthesis (HLS) tools, which can produce less optimal results [8]. We present an alternative approach of developing such hardware in the Chisel Hardware Construction Language (HCL) [9].…”