Applying the Reconfigurable Computing Environment Concept to the Deep Neural Network Accelerators Development

Shatravin, Vladislav; Shashev, Dmitriy; Shidlovskiy, Stanislav

doi:10.1109/icit52682.2021.9491771

Cited by 3 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The characteristics of the proposed SoftMax implementation were evaluated through simulations on Field-Programmable Gate Arrays (FPGA). FPGAs are widely used in the research and development of state-of-the-art computing systems, and numerous useful software tools are available [6,7,23].…”

Section: Resultsmentioning

confidence: 99%

“…The literature review allowed us to identify the following set of operations: "signal source" (SRC), "signal transfer" (TRS), "multiply and accumulate" (MAC), "parametric ReLU" (PRL) "maximum" (MAX), "minimum" (MIN), "gate" (GAT), "union" (U), "delay" (DEL), and "block" (BLK) (Figure 3). These operations are sufficient for implementing the key layers of neural networks (dense, convolution, pooling) as well as activations (sigmoid, tanh, ELU, and other) [21,[23][24][25]. This paper focuses on the implementation of SoftMax activation.…”

Section: Implementation Of Neural Network In Rcementioning

confidence: 99%

“…Thus, the desired algorithms must be implemented as a spatially distributed composition of the given operations. In the research, we propose the PE with 10 simple operations: signal source, signal transfer, multiply-accumulate, parametric ReLU, maximum, minimum, gate, union, delay, and block [21,[23][24][25]. There is currently no implementation of SoftMax activation based on spatial distribution and these operations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Implementation of the SoftMax Activation for Reconfigurable Neural Network Hardware Accelerators

Shatravin,

Shashev,

Shidlovskiy

2023

Applied Sciences

Self Cite

View full text Add to dashboard Cite

In recent decades, machine-learning algorithms have been extensively utilized to tackle various complex tasks. To achieve the high performance and efficiency of these algorithms, various hardware accelerators are used. Typically, these devices are specialized for specific neural network architectures and activation functions. However, state-of-the-art complex autonomous and mobile systems may require different algorithms for different tasks. Reconfigurable accelerators can be used to resolve this problem. They possess the capability to support diverse neural network architectures and allow for significant alterations to the implemented model at runtime. Thus, a single device can be used to address entirely different tasks. Our research focuses on dynamically reconfigurable accelerators based on reconfigurable computing environments (RCE). To implement the required neural networks on such devices, their algorithms need to be adapted to the homogeneous structure of RCE. This article proposes the first implementation of the widely used SoftMax activation for hardware accelerators based on RCE. The implementation leverages spatial distribution and incorporates several optimizations to enhance its performance. The timing simulation of the proposed implementation on FPGA shows a high throughput of 1.12 Gbps at 23 MHz. The result is comparable to counterparts lacking reconfiguration capability. However, this flexibility comes at the expense of the increased consumption of logic elements.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Implementation Of Neural Network In Rcementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Implementation of the SoftMax Activation for Reconfigurable Neural Network Hardware Accelerators

Shatravin,

Shashev,

Shidlovskiy

2023

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

Application of the Piecewise Linear Approximation Method in a Hardware Accelerators of a Neural Networks Based on a Reconfigurable Computing Environments

Shatravin

Shashev

2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Sigmoid Activation Implementation for Neural Networks Hardware Accelerators Based on Reconfigurable Computing Environments for Low-Power Intelligent Systems

2022

View full text Add to dashboard Cite

The remarkable results of applying machine learning algorithms to complex tasks are well known. They open wide opportunities in natural language processing, image recognition, and predictive analysis. However, their use in low-power intelligent systems is restricted because of high computational complexity and memory requirements. This group includes a wide variety of devices, from smartphones and Internet of Things (IoT)smart sensors to unmanned aerial vehicles (UAVs), self-driving cars, and nodes of Edge Computing systems. All of these devices have severe limitations to their weight and power consumption. To apply neural networks in these systems efficiently, specialized hardware accelerators are used. However, hardware implementation of some neural network operations is a challenging task. Sigmoid activation is popular in the classification problem and is a notable example of such a complex operation because it uses division and exponentiation. The paper proposes efficient implementations of this activation for dynamically reconfigurable accelerators. Reconfigurable computing environments (RCE) allow achieving reconfigurability of accelerators. The paper shows the advantages of applying such accelerators in low-power systems, proposes the centralized and distributed hardware implementations of the sigmoid, presents comparisons with the results of other studies, and describes application of the proposed approaches to other activation functions. Timing simulations of the developed Verilog modules show low delay (14–18.5 ns) with acceptable accuracy (average absolute error is 0.004).

show abstract

Applying the Reconfigurable Computing Environment Concept to the Deep Neural Network Accelerators Development

Cited by 3 publications

References 9 publications

Implementation of the SoftMax Activation for Reconfigurable Neural Network Hardware Accelerators

Implementation of the SoftMax Activation for Reconfigurable Neural Network Hardware Accelerators

Application of the Piecewise Linear Approximation Method in a Hardware Accelerators of a Neural Networks Based on a Reconfigurable Computing Environments

Sigmoid Activation Implementation for Neural Networks Hardware Accelerators Based on Reconfigurable Computing Environments for Low-Power Intelligent Systems

Contact Info

Product

Resources

About