Optimizing Temporal Convolutional Network inference on FPGA-based accelerators

Abstract

Convolutional Neural Networks are extensively used in a wide range of applications, commonly including computer vision tasks like image and video classification, recognition and segmentation. Recent research results demonstrate that multilayer (deep) network involving mono-dimensional convolutions and dilation can be effectively used in time series and sequences classification and segmentation, as well as in tasks involving sequence modelling. These structures, commonly referred to as Temporal Convolutional Networks (TCNs), have been demonstrated to consistently outperform Recurrent Neural Networks in terms of accuracy and training time [1]. While FPGA-based inference accelerators for classic CNNs are widespread, literature is lacking in a quantitative evaluation of their usability on inference for TCN models. In this paper we present such an evaluation, considering a CNN accelerator with specific features supporting TCN kernels as a reference and a set of state-ofthe-art TCNs as benchmark. Experimental results show that, during TCN execution, operational intensity can be critical for the overall performance. We propose a convolution scheduling based on batch processing that can boost efficiency up to 96% of theoretical peak performance. Overall we can achieve up to 111,8 GOPS/s and a power efficiency of 33,9 GOPS/s/W on an Ultrascale+ ZU3EG (up to 10x speedup and 3x power efficiency improvement with respect to pure software implementation).

Dilate-Invariant Temporal Convolutional Network for Real-Time Edge Applications

“…The increasing interest in TCNs encourages the exploration of the embedded computation flow of such type of networks, such as system-level mapping to FPGAs [27], [28], or generally with dilated convolution such as in [29] and [30]. The work in [27] addresses the possibility of mapping TCNs to existing CNN accelerators, by proposing a convolution scheduling based on batch processing. The TCN architecture is based on NEURAghe [31], it consists of Sum-of-Product units, each containing a group of DSPs.…”

Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Codesign

“…For sparse and hybrid DNNs, Huang et al [89] propose a configurable inference engine capable of processing different sizes of sparse DNNs, while HybridDNN [90] has a hybrid architecture composed of spatial/Winograd convolution processing elements. Additionally, Carreras et al [91] present an enriched architectural template supporting efficient TCNs, together with an algorithm for optimal execution/scheduling of data-transfers to boost the implementation performance.…”