FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review

Shawahna, Ahmad; Sait, Sadiq M.; El-Maleh, Aiman H.

doi:10.1109/access.2018.2890150

Cited by 346 publications

(146 citation statements)

References 162 publications

Supporting

Mentioning

140

Contrasting

Unclassified

Order By: Relevance

“…e number of MAC operations can reach the magnitude of billion [42]. Additionally, a single deep learning network can contain over a million parameters [43]. As a result, deep learning proposes high demands on processing ability, memory capacity, and energy efficiency.…”

Section: Challenges To Be Investigatedmentioning

confidence: 99%

Deep Learning on Computational-Resource-Limited Platforms: A Survey

Chen

Zhang

et al. 2020

Mobile Information Systems

View full text Add to dashboard Cite

Nowadays, Internet of Things (IoT) gives rise to a huge amount of data. IoT nodes equipped with smart sensors can immediately extract meaningful knowledge from the data through machine learning technologies. Deep learning (DL) is constantly contributing significant progress in smart sensing due to its dramatic superiorities over traditional machine learning. The promising prospect of wide-range applications puts forwards demands on the ubiquitous deployment of DL under various contexts. As a result, performing DL on mobile or embedded platforms is becoming a common requirement. Nevertheless, a typical DL application can easily exhaust an embedded or mobile device owing to a large amount of multiply and accumulate (MAC) operations and memory access operations. Consequently, it is a challenging task to bridge the gap between deep learning and resource-limited platforms. We summarize typical applications of resource-limited deep learning and point out that deep learning is an indispensable impetus of pervasive computing. Subsequently, we explore the underlying reasons for the high computational overhead of DL through reviewing the fundamental concepts including capacity, generalization, and backpropagation of a neural network. Guided by these concepts, we investigate on principles of representative research works, as well as three types of solutions: algorithmic design, computational optimization, and hardware revolution. In pursuant to these solutions, we identify challenges to be addressed.

show abstract

Section: Challenges To Be Investigatedmentioning

confidence: 99%

Deep Learning on Computational-Resource-Limited Platforms: A Survey

Chen

Zhang

et al. 2020

Mobile Information Systems

View full text Add to dashboard Cite

show abstract

“…Nowadays FPGAs are used in a wide range of applications. They were historically used for high-density and parallel computations as well as neural network accelerators [34], but also for power electronics applications [2]. Typically, FPGAs are designed with a hard-macro approach, repeating the optimized basic block so as to realize a huge-sized IP with enormous computational capability.…”

Section: B Fpgasmentioning

confidence: 99%

A Fully Programmable eFPGA-Augmented SoC for Smart Power Applications

Renzini

Mucci

Rossi

et al. 2020

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

This paper proposes a reconfigurable System-on-Chip (SoC) for smart power applications. The system is composed of an ultra-low-power microcontroller for standard software programmability, coupled to an embedded-FPGA (eFPGA) to perform control-driven applications and lightweight digital signal processing, at a lower power consumption and higher responsiveness than with processor-based execution. To the best of the authors' knowledge this is the first heterogeneous reconfigurable SoC targeting smart power applications. The SoC targets BCD technologies integrating Bipolar, CMOS and DMOS devices, typically featuring a small amount of metal layers when compared to traditional CMOS technologies. The added value of the proposed system is that the digital system is fully synthesizable, since the eFPGA is based on a soft-core approach. The paper presents the results of integrating an eFPGA with a computational capability of 1k equivalent gates in STMicroelectronics 90 nm BCD technology featuring five metal layers and high-k transistors. We benchmarked our architecture on a wide range of applications relevant to the smart power domain. eFPGA integration in SoCs introduces a 20-27% area overhead, but has a straightforward benefit in terms of energy consumption which proves reduced from about 10× to 800×. In terms of latency, the eFPGA implementation allows a gain from 8× to 145× comparing the pure cycles count.

show abstract

“…Based on the previous pivot rotation formula, it was possible to successfully compute a dynamic mapping algorithm (Equation (19)) that accurately builds an aerial view over the UAV's flown area.…”

Section: Aerial Mappingmentioning

confidence: 99%

“…The execution of algorithms for terrain classification is computationally heavy and can lead to lower performance than desired. Field Programmable Gate Array (FPGA) implementations have been used to accelerate the execution of algorithms due to their maximization of parallel processing and lower energy consumption [19]. This features allow FPGA implementations to achieve faster execution times when compared to computer vision software libraries, such as OpenCV, or high performance interactive software for numerical computation, such as MATLAB [20,21].…”

Section: Introductionmentioning

confidence: 99%

Static and Dynamic Algorithms for Terrain Classification in UAV Aerial Imagery

et al. 2019

View full text Add to dashboard Cite

The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV's mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV's downwash effect: Static textures-Gray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic textures-Optical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.Several methods for terrain classification have been proposed in different works. Texture algorithms, such as those proposed in [2][3][4][5], have been widely recommended to emphasize the high and low frequencies of the images, supporting image classification. Other algorithms use color information to classify terrains, such as presented in [6], which is able to distinguish four different terrain types within an image. During this process, each channel's pixel is divided by the square root of its own three channels intensity. The final result will emphasize the color that most represents the terrain type (eg, blue for water). Additionally, frequency domain [7,8], segmentation [6,9,10], bayesian network [11], and Hyperspectal Images [12] can also be used in terrain classification.Other types of sensors such as LiDAR [13][14][15][16] can complement the classification decision. Algorithms that use laser scanners proved to be qualified to accurately distinguish between water and non-water terrains [13][14][15][16]. However, shallow water terrains increase the decision error due to laser reflection, which leads to a misclassification as non-water terrain.Although prior research work has proposed many good solutions for terrain classification, there is still a gap regarding the study of dynamic terrain. The previously mentioned algorithms suffer from a high sensitivity to changes in the environment, mainly due to changes in brightness, color and texture. Recent works [17,18] ...

show abstract

FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review

Cited by 346 publications

References 162 publications

Deep Learning on Computational-Resource-Limited Platforms: A Survey

Deep Learning on Computational-Resource-Limited Platforms: A Survey

A Fully Programmable eFPGA-Augmented SoC for Smart Power Applications

Static and Dynamic Algorithms for Terrain Classification in UAV Aerial Imagery

Contact Info

Product

Resources

About