FINN-L: Library Extensions and Design Trade-Off Analysis for Variable Precision LSTM Networks on FPGAs

Abstract

It is well known that many types of artificial neural networks, including recurrent networks, can achieve a high classification accuracy even with low-precision weights and activations. The reduction in precision generally yields much more efficient hardware implementations in regards to hardware cost, memory requirements, energy, and achievable throughput. In this paper, we present the first systematic exploration of this design space as a function of precision for Bidirectional Long Short-Term Memory (BiLSTM) neural network. Specifically, we include an in-depth investigation of precision vs. accuracy using a fully hardware-aware training flow, where during training quantization of all aspects of the network including weights, input, output and in-memory cell activations are taken into consideration. In addition, hardware resource cost, power consumption and throughput scalability are explored as a function of precision for FPGA-based implementations of BiLSTM, and multiple approaches of parallelizing the hardware. We provide the first open source HLS library extension of FINN [1] for parameterizable hardware architectures of LSTM layers on FPGAs which offers full precision flexibility and allows for parameterizable performance scaling offering different levels of parallelism within the architecture. Based on this library, we present an FPGA-based accelerator for BiLSTM neural network designed for optical character recognition, along with numerous other experimental proof points for a Zynq UltraScale+ XCZU7EV MPSoC within the given design space.

A Review of Binarized Neural Networks

“…These types of architectures are well suited for constrained resource designs where high throughput is not needed. The feedback nature of layer processors also make them well suited for RNNs, as seen in [69].…”

“…This type of work flow is called High Level Synthesis (HLS). HLS has been a major component of the research done with BNNs on FPGAs [35,[38][39][40]54,56,63,68,69,72].…”

“…The trained model is then used by FINN to produce code for the BNN which it synthesizes into a hardware design by Xilinx's HLS tool. The FINN tool received an extension allowing it to work with BNN topologies for LSTMs [69]. Xilinx Research labs also extended the capabilities of FINN by allowing it to work with multi-bit quantized networks, not just with BNNs [54].…”

Hardware resources analysis of BNNs splitting for FARD-based multi-FPGAs Distributed Systems

“…Finally, other works, such as [10], [11], extended FINN in the past. The former proposes an extension to the original version of the framework with support for arbitrary precision and more flexibility in the end architecture and target platforms, including hardware cost estimation for given devices.…”

Efficient Weight Reuse for Large LSTMs

“…There has been previous work [7,8,9,10] with FPGA based implementations such that all the weights are stored in the on-chip memory, but this is expensive and limits the size of models that can be deployed. When the RNN model is too large that the weights need to be stored on an external DRAM, it is not efficient because the fetched weights are typically used only once for each output computation.…”