LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications

Abstract

Deployment of deep neural networks for applications that require very high throughput or extremely low latency is a severe computational challenge, further exacerbated by inefficiencies in mapping the computation to hardware. We present a novel method for designing neural network topologies that directly map to a highly efficient FPGA implementation. By exploiting the equivalence of artificial neurons with quantized inputs/outputs and truth tables, we can train quantized neural networks that can be directly converted to a netlist of truth tables, and subsequently deployed as a highly pipelinable, massively parallel FPGA circuit. However, the neural network topology requires careful consideration since the hardware cost of truth tables grows exponentially with neuron fan-in. To obtain smaller networks where the whole netlist can be placed-and-routed onto a single FPGA, we derive a fan-in driven hardware cost model to guide topology design, and combine high sparsity with low-bit activation quantization to limit the neuron fan-in. We evaluate our approach on two tasks with very high intrinsic throughput requirements in high-energy physics and network intrusion detection. We show that the combination of sparsity and low-bit activation quantization results in high-speed circuits with small logic depth and low LUT cost, demonstrating competitive accuracy with less than 15 ns of inference latency and throughput in the hundreds of millions of inferences per second.

Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors

“…The same group have also developed a library for quantization-aware training, Brevitas [46], based on PyTorch model formats. The Log-icNets design flow [47], also from Xilinx Research Labs, allows for the training of quantized DNNs that map to highly efficient Xilinx FPGA implementations. A comparison between the approach presented here and Logic-Nets is provided in Section VII.…”

“…Further, we compare the results obtained using the QKeras and hls4ml workflow to LogicNets [47]; another work on extreme low-latency, low-resource, fully-unfolded (II=1) FPGA implementations. The metrics are those quoted in Table III.…”

Classification Functions for Handwritten Digit Recognition

“…Since, each unit requires only one BRAM access, it is faster than FINN-R. Also, it requires lower power, since our network requires fewer BRAMs than FINN-R. Note that this comparison excludes the softmax part [20].…”

Logic Shrinkage

“…With LUTNet, we reported area efficiency improvements of around 2× over ReBNet [3], the state-of-the-art BNN at the time, for problems of widely varying scale. More recent tools, including NullaNet [12] and LogicNets [18], also generate small LUTs as core components, but LUTNet remains unique in directly exposing a netlist's LUTs as differentiable functions trainable via stochastic gradient descent.…”

“…RELATED WORK 2.1 FPGA-Tailored DNN ArchitecturesLUT-based DNN inference accelerators have been shown to achieve remarkable performance when deployed on FPGAs. NullaNet[12] and LogicNets[18] were conceived with small-scale classification tasks in mind, for which they reached latency in the tens of nanoseconds and throughput in the hundreds of millions of samples per second. Going beyond FPGA-tailored network design, our previously proposed LUTNet topologies can be trained via stochastic gradient descent[20].…”