Moving Learning Machine towards Fast Real-Time Applications: A High-Speed FPGA-Based Implementation of the OS-ELM Training Algorithm

Abstract

Alfredo.Rosado@uv.es (A.R.-M.); manuel.bataller@uv.es (M.B.-M.); Juan.Barrios@uv.es (J.B.-A.); juan.guerrero@uv.es (J.F.G.-M.)Abstract: Currently, there are some emerging online learning applications handling data streams in real-time. The On-line Sequential Extreme Learning Machine (OS-ELM) has been successfully used in real-time condition prediction applications because of its good generalization performance at an extreme learning speed, but the number of trainings by a second (training frequency) achieved in these continuous learning applications has to be further reduced. This paper proposes a performance-optimized implementation of the OS-ELM training algorithm when it is applied to real-time applications. In this case, the natural way of feeding the training of the neural network is one-by-one, i.e., training the neural network for each new incoming training input vector. Applying this restriction, the computational needs are drastically reduced. An FPGA-based implementation of the tailored OS-ELM algorithm is used to analyze, in a parameterized way, the level of optimization achieved. We observed that the tailored algorithm drastically reduces the number of clock cycles consumed for the training execution up to approximately the 1%. This performance enables high-speed sequential training ratios, such as 14 KHz of sequential training frequency for a 40 hidden neurons SLFN, or 180 Hz of sequential training frequency for a 500 hidden neurons SLFN. In practice, the proposed implementation computes the training almost 100 times faster, or more, than other applications in the bibliography. Besides, clock cycles follows a quadratic complexity O(Ñ 2 ), withÑ the number of hidden neurons, and are poorly influenced by the number of input neurons. However, it shows a pronounced sensitivity to data type precision even facing small-size problems, which force to use double floating-point precision data types to avoid finite precision arithmetic effects. In addition, it has been found that distributed memory is the limiting resource and, thus, it can be stated that current FPGA devices can support OS-ELM-based on-chip learning of up to 500 hidden neurons. Concluding, the proposed hardware implementation of the OS-ELM offers great possibilities for on-chip learning in portable systems and real-time applications where frequent and fast training is required.Electronics 2018, 7, 308 2 of 23 learning of neural networks for the prediction of future opponent robot coordinates; ref.[3] designed an ASIC on-chip learning to learn and extract features existing in input datasets, intended to embedded vision applications; or [4], that implemented a real-time classifier for neurological signals.The Extreme Learning Machine (ELM) algorithm possesses many aspects that makes it suitable for any real-time or custom hardware implementation. It has a reduced and fixed training time along with an extremely fast learning speed that allows determinism in the computation time and, thus, a great advantage compared to previous well-known training...

A Cost-Efficient Software Based Router and Traffic Generator for Simulation and Testing of IP Network

“…The advantage of the developed traffic generator is that, unlike the known ones, it allows you to control the generated flow without the need for synchronize the input and output interfaces and to determine the packets delay of flow. It allows you to evaluate the parameters of service of hardware and software telecommunications facilities of any purpose [53][54][55][56][57][58][59].…”

Real-Time Localization of Epileptogenic Foci EEG Signals: An FPGA-Based Implementation

“…This application needs a real-time implementation, and could be used as a help-decision tool by neurosurgeons to refine the localization of an epileptogenic area during resective epilepsy surgery. Note that the recent technology is mature enough to implement machine learning processes in real-time [27,28].…”

A Novel Systolic Parallel Hardware Architecture for the FPGA Acceleration of Feedforward Neural Networks

“…• In a Virtex-7 FPGA implementation, SYMPA architecture accelerates up to x256 times with respect to the PC implementation and can perform up to 1980 GOPS when using 3600 neuron units per layer. Despite some works demonstrate the feasibility of on-chip learning [19], [20], embedded learning is not considered in this work since weights are generally calculated using off-line procedures (backpropagation, ELM, etc.). Once calculated, the weight values are loaded to the FPGA internal memory.…”