Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

Abstract

Currently, Machine Learning (ML) is becoming ubiquitous in everyday life. Deep Learning (DL) is already present in many applications ranging from computer vision for medicine to autonomous driving of modern cars as well as other sectors in security, healthcare, and finance. However, to achieve impressive performance, these algorithms employ very deep networks, requiring a significant computational power, both during the training and inference time. A single inference of a DL model may require billions of multiply-and-accumulated operations, making the DL extremely compute-and energy-hungry. In a scenario where several sophisticated algorithms need to be executed with limited energy and low latency, the need for cost-effective hardware platforms capable of implementing energy-efficient DL execution arises. This paper first introduces the key properties of two brain-inspired models like Deep Neural Network (DNN), and Spiking Neural Network (SNN), and then analyzes techniques to produce efficient and high-performance designs. This work summarizes and compares the works for four leading platforms for the execution of algorithms such as CPU, GPU, FPGA and ASIC describing the main solutions of the state-of-the-art, giving much prominence to the last two solutions since they offer greater design flexibility and bear the potential of high energy-efficiency, especially for the inference process. In addition to hardware solutions, this paper discusses some of the important security issues that these DNN and SNN models may have during their execution, and offers a comprehensive section on benchmarking, explaining how to assess the quality of different networks and hardware systems designed for them.

Low-Power Detection and Classification for In-Sensor Predictive Maintenance Based on Vibration Monitoring

“…However, such models usually require high computational complexity and memory footprint, so that they are not well suited for resource constrained embedded systems. The use of cloud computing is inconvenient when operational critical apparatus must be monitored, due to the low reliability and high latency of remote connections which requires enough bandwidth to guarantee real-time operations; general purpose platforms, using CPUs and GPUs have got silicon sizes, prices and energy costs which are incompatible with the integration into the apparatus to be monitored [5]. Similar limitations affect devoted processors, such as the Xilinx Deep Learning Processor Unit (DPU) core [20], introduced to accelerate CNN inference on FPGAs.…”

“…Recent DL approaches are based on Convolutional Neural Networks (CNN), Generative Adversarial Network (GAN), Recurrent Neural Networks (RNN), etc, each one having its own strengths and weaknesses [1]. However, deploying such networks in PdM systems is very challenging and costly due to the very high computational complexity, memory requirements of those models, and power consumption, which do not meet the needs for always-on monitoring of the apparatus [4], [5]. In this context, Auto-Encoders (AE) [6] are an obvious choice since they combine the use of relatively shallow networks with the possibility of unsupervised training, particularly interesting when the availability of labelled faults data is difficult.…”

Framework for Assessing Ethical Aspects of Algorithms and Their Encompassing Socio-Technical System

“…A commonly overlooked issue with advanced machine learning methods is their energy consumption: model training and development likely make up a substantial portion of the greenhouse gas emissions because of excessively large models used, for example, in natural language processing [30]. An important solution direction is to use special hardware designed for energy-efficient learning [31,32].…”

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

“…This increased complexity hinders the deployment of advanced NNs (DNNs and SNNs) on resourceconstrained edge devices [4]. Therefore, optimizations at different system layers (i.e., HW and SW) are necessary to enable the use of advanced NNs at the edge [2]. Besides performance and energy efficiency, reliability and security aspects are also important to ensure Fig.…”