Hardware implementation of real-time Extreme Learning Machine in FPGA: Analysis of precision, resource occupation and performance

Francés-Víllora, Jose V.; Muñoz, Andrés Pedreño; Martínez-Villena, José M.; Bataller-Mompeán, Manuel; Guerrero, Juan; Węgrzyn, Marek

doi:10.1016/j.compeleceng.2016.02.007

Cited by 47 publications

(22 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This impact is achieved from the simplification of the update computation of P k+1 and β matrices, Equations (16) and (17), which enables to use only matrix by vector or vector by vector multiplications for the update computation. Otherwise, when chunks of size larger than one are used, the update computation needs aÑ ×Ñ matrix inversion, Equation (15), which implies computing a QR decomposition followed by a Triangular Matrix Inversion, which is computationally tough [48]. Thus, the proposed one-by-one training greatly simplifies computation.…”

Section: Discussionmentioning

confidence: 99%

“…The latter case requires other matrix operations, but the most consuming matrix operation is theÑ ×Ñ matrix inversion, and hence, the number of execution cycles for the matrix inversion may constitute a rough estimation (downward estimation) for this case. With comparison purposes, this value can be obtained from [48], where an optimal FPGA-based implementation of the ELM training was implemented, the number of training clock cycles was expressed analytically, and almost all the computational effort involved the inversion computation. Table 2 compares the number of clock cycles needed for the OS-ELM training using the proposed one-by-one training strategy and chunk feeding (downward estimation obtained from [48]).…”

Section: Discussionmentioning

confidence: 99%

“…With comparison purposes, this value can be obtained from [48], where an optimal FPGA-based implementation of the ELM training was implemented, the number of training clock cycles was expressed analytically, and almost all the computational effort involved the inversion computation. Table 2 compares the number of clock cycles needed for the OS-ELM training using the proposed one-by-one training strategy and chunk feeding (downward estimation obtained from [48]). The comparison is achieved for different number of neurons in the hidden layer.…”

Section: Discussionmentioning

confidence: 99%

“…** Number of clock cycles estimated for the OS-ELM training using a hardware implementation following the chunk feeding strategy. The estimation comes from [48]. As the reference design use fixed-point arithmetic, a 24-bits length has been used to obtain these data.…”

Section: Discussionmentioning

confidence: 99%

See 3 more Smart Citations

Moving Learning Machine towards Fast Real-Time Applications: A High-Speed FPGA-Based Implementation of the OS-ELM Training Algorithm

et al. 2018

Self Cite

View full text Add to dashboard Cite

Alfredo.Rosado@uv.es (A.R.-M.); manuel.bataller@uv.es (M.B.-M.); Juan.Barrios@uv.es (J.B.-A.); juan.guerrero@uv.es (J.F.G.-M.)Abstract: Currently, there are some emerging online learning applications handling data streams in real-time. The On-line Sequential Extreme Learning Machine (OS-ELM) has been successfully used in real-time condition prediction applications because of its good generalization performance at an extreme learning speed, but the number of trainings by a second (training frequency) achieved in these continuous learning applications has to be further reduced. This paper proposes a performance-optimized implementation of the OS-ELM training algorithm when it is applied to real-time applications. In this case, the natural way of feeding the training of the neural network is one-by-one, i.e., training the neural network for each new incoming training input vector. Applying this restriction, the computational needs are drastically reduced. An FPGA-based implementation of the tailored OS-ELM algorithm is used to analyze, in a parameterized way, the level of optimization achieved. We observed that the tailored algorithm drastically reduces the number of clock cycles consumed for the training execution up to approximately the 1%. This performance enables high-speed sequential training ratios, such as 14 KHz of sequential training frequency for a 40 hidden neurons SLFN, or 180 Hz of sequential training frequency for a 500 hidden neurons SLFN. In practice, the proposed implementation computes the training almost 100 times faster, or more, than other applications in the bibliography. Besides, clock cycles follows a quadratic complexity O(Ñ 2 ), withÑ the number of hidden neurons, and are poorly influenced by the number of input neurons. However, it shows a pronounced sensitivity to data type precision even facing small-size problems, which force to use double floating-point precision data types to avoid finite precision arithmetic effects. In addition, it has been found that distributed memory is the limiting resource and, thus, it can be stated that current FPGA devices can support OS-ELM-based on-chip learning of up to 500 hidden neurons. Concluding, the proposed hardware implementation of the OS-ELM offers great possibilities for on-chip learning in portable systems and real-time applications where frequent and fast training is required.Electronics 2018, 7, 308 2 of 23 learning of neural networks for the prediction of future opponent robot coordinates; ref.[3] designed an ASIC on-chip learning to learn and extract features existing in input datasets, intended to embedded vision applications; or [4], that implemented a real-time classifier for neurological signals.The Extreme Learning Machine (ELM) algorithm possesses many aspects that makes it suitable for any real-time or custom hardware implementation. It has a reduced and fixed training time along with an extremely fast learning speed that allows determinism in the computation time and, thus, a great advantage compared to previous well-known training...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Moving Learning Machine towards Fast Real-Time Applications: A High-Speed FPGA-Based Implementation of the OS-ELM Training Algorithm

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Field Programmable Gate Array (FPGA) is a silicon chip that can be reprogrammed to perform specific functions or applications instead of running through software application [6]. In 2016, [7] proposed an FPGA implementation of a real-time extreme learning machine. Another variant of ELM namely Online Sequential Extreme Learning Machine (OS-ELM) is also implemented using FPGA [8][9].…”

mentioning

confidence: 99%

Accelerating Extreme Learning Machine on FPGA by Hardware Implementation of Given Rotation-QRD

Tan

Ismail

Ooi

et al. 2019

IJIE

View full text Add to dashboard Cite

Currently, Extreme Learning Machine (ELM) is one of the research trends in the machine learning field due to its remarkable performances in terms of complexity and computational speed. However, the big data era and the limitations of general-purpose processor cause the increasing of interest in hardware implementation of ELM in order to reduce the computational time. Hence, this work presents the hardware-software co-design of ELM to improve the overall performances. In the co-design paradigm, one of the important components of ELM, namely Given Rotation-QRD (GR-QRD) is developed as a hardware core. Field Programmable Gate Array (FPGA) is chosen as the platform for ELM implementation due to its reconfigurable capability and high parallelism. Moreover, the learning accuracy and computational time would be used to evaluate the performances of the proposed ELM design. Our experiment has shown that GR-QRD accelerator helps to reduce the computational time of ELM training by 41.75% while maintaining the same training accuracy in comparison to pure software of ELM.

show abstract

Efficient hardware‐accelerated pseudoinverse computation through algorithm restructuring for parallelization in high‐level synthesis

Tan

Ooi

Choo

et al. 2021

Circuit Theory & Apps

View full text Add to dashboard Cite

This paper describes a fast and efficient hardware-accelerated pseudoinverse computation through algorithm restructuring and leveraging FPGA synthesis directives for parallelism prior to high-level synthesis (HLS). The algorithm, which is composed of modified Gram-Schmidt QR decomposition (MGS-QRD), triangular matrix inversion (TMI), and matrix multiplication (MM), is synthesized and implemented on a field-programmable gate array (FPGA). MGS-QRD is restructured and augmented with parallelism directives prior to synthesizing the algorithm, which yielded an MGS-QRD hardware accelerator with high throughput. Modifications to the current TMI algorithm were also proposed, in which the removal of redundant computational tasks was done in order to speed up overall operation. Data dependencies in the MM algorithm were carefully considered such that appropriate parallelism directives were inserted, and matching the data flow of MM with MGS-QRD and TMI modules was also performed to accelerate the pseudoinverse computation. The results showed that the proposed pseudoinverse module is better than the naïve implementation which is composed of existing MGS-QRD, TMI and a standard MM in terms of maximum frequency (1.24Â speedup), hardware resources (48% of reduction of DSP usage), latency (23% reduction), and throughput (62% increase).

show abstract

Hardware implementation of real-time Extreme Learning Machine in FPGA: Analysis of precision, resource occupation and performance

Cited by 47 publications

References 15 publications

Moving Learning Machine towards Fast Real-Time Applications: A High-Speed FPGA-Based Implementation of the OS-ELM Training Algorithm

Moving Learning Machine towards Fast Real-Time Applications: A High-Speed FPGA-Based Implementation of the OS-ELM Training Algorithm

Accelerating Extreme Learning Machine on FPGA by Hardware Implementation of Given Rotation-QRD

Efficient hardware‐accelerated pseudoinverse computation through algorithm restructuring for parallelization in high‐level synthesis

Contact Info

Product

Resources

About