FPGA Implementations of Feed Forward Neural Network by using Floating Point Hardware Accelerators

A Neural Network Embedded System for Real-time Estimation of Muscle Forces

“…Two alternatives were used: the full-precision activation function, computed using math.h C library, and a polynomial interpolation of the activation function, pre-computed in Matlab, composed by a fit of five 2 nd order polynomials [18]. The second solution is obviously a tradeoff Table 2: Code configurations for best performance and best precision.…”

“…Networks were trained and validated in Matlab® environment according to the methodology described in [16,35]. To enhance the embedded performance of the NN, according to [18,19], a speedup for the activation function of the hidden neurons was obtained by computing it through a 2nd degree polynomial interpolating function.…”

“…When implementing this algorithm in an embedded environment, the limited computational capabilities calls for a trade-off among precision, memory footprint, and computational cost. Different studies tested the embedded implementation of NNs to achieve optimal results, either by re-arranging the operations required to compute the linear part of the NN [17] to fully exploit pipelining, or by speeding up the costly non-linear activation function through different numerical approximation [18,19,20,21,22,23,24].…”

On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review

“…A variation of the simple LUT is the RA-LUT (Range Addressable LUT), where each sample corresponds not only to a specific point in the domain, but to a neighborhood of the point. In [39] the authors propose a comparison between two FPGA architectures which uses floating-point accelerators based on RA-LUT to compute fast AFs. The first solution, refined from the one proposed in [40, 41], implements the NN on a soft processor and computes the AFs through a smartly spaced RA-LUT.…”

“…Combination of LUT and PWL approximation is also used in [56, 57], where different authors investigate the approximation in fixed point for the synthesis of exponential and sigmoid AFs. In [39, 58] authors propose a piecewise II-degree polynomial approximation of the activation function for both the sigmoid and the hyperbolic tangent AFs. Performance, resources required, and precision degradation are compared to full-precision and RA-LUT solutions.…”