Exploring NEURAghe: A Customizable Template for APSoC-Based CNN Inference at the Edge

Meloni, Paolo; Loi, Daniela; Deriu, Gianfranco; Carreras, Marco; Conti, Francesco; Capotondi, Alessandro; Rossi, Davide

doi:10.1109/les.2019.2947312

Cited by 6 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We do not account for contention on the off-chip memory, since previous experiments have shown that the effect of this issue is limited. [51] shows this for the case of multiple NEURAghe instances insisting on the same DDR memory. We also assume, as in most of the approaches in the literature, communication with the host to be asynchronous, thus we do not consider the host CPU intervention to become a bottleneck.…”

Section: Cnn Metric Aggregationmentioning

confidence: 82%

See 1 more Smart Citation

ALOHA: A Unified Platform-Aware Evaluation Method for CNNs Execution on Heterogeneous Systems at the Edge

Busia¹,

Minakova

Stefanov

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

CNN design and deployment on embedded edge-processing systems is an error-prone and effort-hungry process, that poses the need for accurate and effective automated assisting tools. In such tools, pre-evaluating the platform-aware CNN metrics such as latency, energy cost, and throughput is a key requirement for successfully reaching the implementation goals imposed by use-case constraints. Especially when more complex parallel and heterogeneous computing platforms are considered, currently utilized estimation methods are inaccurate or require a lot of characterization experiments and efforts. In this paper, we propose an alternative method, designed to be flexible, easy to use, and accurate at the same time. Considering a modular platform and execution model that adequately describes the details of the platform and the scheduling of different CNN operators on different platform processing elements, our method captures precisely operations and data transfers and their deployment on computing and communication resources, significantly improving the evaluation accuracy. We have tested our method on more than 2000 CNN layers, targeting an FPGA-based accelerator and a GPU platform as reference example architectures. Results have shown that our evaluation method increases the estimation precision by up to 5× for execution time, and by 2× for energy, compared to other widely used analytical methods. Moreover, we assessed the impact of the improved platform-awareness on a set of neural architecture search experiments, targeting both hardware platforms, and enforcing 2 sets of latency constraints, performing 5 trials on each search space, for a total number of 20 experiments. The predictability is improved by 4×, reaching, with respect to alternatives, selection results clearly more similar to those obtained with on-hardware measurements.

show abstract

Section: Cnn Metric Aggregationmentioning

confidence: 82%

“…Despite systems with multiple accelerators being hardly available in the embedded domain, some research efforts have demonstrated that such a design technique can be useful [51], [52]. Thus our aggregation methodology enables the estimation to take into account an arbitrary number of processing elements.…”

Section: Cnn Metric Aggregationmentioning

confidence: 99%

ALOHA: A Unified Platform-Aware Evaluation Method for CNNs Execution on Heterogeneous Systems at the Edge

Busia¹,

Minakova

Stefanov

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In Table VIII we report comparative results with state-of-the-art on two well known networks for image classification, ResNet-18 and VGG-16. We compare to other accelerator architectures, [44], [45] and [46], that are implemented on the same kind of hardware and use the same 16-bit data precision. On VGG-16, that exposes quite regular kernel sizes and stride values, our work shows comparable performance with respect to the alternatives.…”

Section: E Comparison To Other Apsoc Based Acceleratorsmentioning

confidence: 99%

“…On VGG-16, that exposes quite regular kernel sizes and stride values, our work shows comparable performance with respect to the alternatives. It executes convolutions slightly faster than [44] and [45] and around 13% slower than [46]. On ResNet-18, which exposes more variable kernel sizes and strides, we can reduce a lot of overheads that must be payed by more static architectures, executing the whole convolution workload 40% faster than [44].…”

Section: E Comparison To Other Apsoc Based Acceleratorsmentioning

confidence: 99%

See 1 more Smart Citation

Optimizing Temporal Convolutional Network inference on FPGA-based accelerators

Carreras¹,

Deriu²,

Raffo³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Convolutional Neural Networks are extensively used in a wide range of applications, commonly including computer vision tasks like image and video classification, recognition and segmentation. Recent research results demonstrate that multilayer (deep) network involving mono-dimensional convolutions and dilation can be effectively used in time series and sequences classification and segmentation, as well as in tasks involving sequence modelling. These structures, commonly referred to as Temporal Convolutional Networks (TCNs), have been demonstrated to consistently outperform Recurrent Neural Networks in terms of accuracy and training time [1]. While FPGA-based inference accelerators for classic CNNs are widespread, literature is lacking in a quantitative evaluation of their usability on inference for TCN models. In this paper we present such an evaluation, considering a CNN accelerator with specific features supporting TCN kernels as a reference and a set of state-ofthe-art TCNs as benchmark. Experimental results show that, during TCN execution, operational intensity can be critical for the overall performance. We propose a convolution scheduling based on batch processing that can boost efficiency up to 96% of theoretical peak performance. Overall we can achieve up to 111,8 GOPS/s and a power efficiency of 33,9 GOPS/s/W on an Ultrascale+ ZU3EG (up to 10x speedup and 3x power efficiency improvement with respect to pure software implementation).

show abstract

An Automated Design Flow for Adaptive Neural Network Hardware Accelerators

Ratto

Máinez

et al. 2023

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

Image and video processing are one of the main driving application fields for the latest technology advancement of computing platforms, especially considering the adoption of neural networks for classification purposes. With the advent of Cyber Physical Systems, the design of devices for efficiently executing such applications became more challenging, due to the increase of the requirements to be considered, of the functionalities to be supported, as well as to the demand for adaptivity and connectivity. Heterogeneous computing and design automation are then turning into essential. The former guarantees a variegated set of features under strict constraints (e.g., by adopting hardware acceleration), and the latter limits development time and cost (e.g., by exploiting model-based design). In this context, the literature is still lacking adequate tooling for the design and management of neural network hardware accelerators, which can be adaptable and customizable at runtime according to the user needs. In this work, a novel almost automated toolchain based on the Open Neural Network eXchange format is presented, allowing the user to shape adaptivity right on the network model and to deploy it on a runtime reconfigurable accelerator. As a proof of concept, a Convolutional Neural Network for human/animal classification is adopted to derive a Field Programmable Gate Array accelerator capable of trading execution time for power by changing the resources involved in the computation. The resulting accelerator, when necessary, can consume 30% less power on each layer, taking about overall 8% more time to classify an image.

show abstract

Exploring NEURAghe: A Customizable Template for APSoC-Based CNN Inference at the Edge

Cited by 6 publications

References 11 publications

ALOHA: A Unified Platform-Aware Evaluation Method for CNNs Execution on Heterogeneous Systems at the Edge

ALOHA: A Unified Platform-Aware Evaluation Method for CNNs Execution on Heterogeneous Systems at the Edge

Optimizing Temporal Convolutional Network inference on FPGA-based accelerators

An Automated Design Flow for Adaptive Neural Network Hardware Accelerators

Contact Info

Product

Resources

About