Towards a more efficient and cost-sensitive extreme learning machine: A state-of-the-art review of recent trend

Alaba, Peter Adeniyi; Popoola, Segun I.; Olatomiwa, Lanre; Akanle, Matthew B.; Ohunakin, Olayinka S.; Adetiba, Emmanuel; Alex, Opeoluwa David; Atayero, Aderemi A.; Daud, Wan Mohd Ashri Wan

doi:10.1016/j.neucom.2019.03.086

Cited by 51 publications

(24 citation statements)

References 152 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work focuses on the use of extreme learning machines (ELM) for computational partial differential equations (PDE). ELM was originally developed in [26,27] for linear classification/regression problems with single hidden-layer feed-forward neural networks, and has since found wide applications in a number of fields; see the reviews in [25,1] and the references therein. Two strategies underlie ELM: (i) random but fixed (non-trainable) hidden-layer coefficients, and (ii) trainable linear output-layer coefficients determined by a linear least squares method or by using the pseudo-inverse of the coefficient matrix (for linear problems).…”

mentioning

confidence: 99%

On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

Dong,

Yang

2021

Preprint

View full text Add to dashboard Cite

We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on [−Rm, Rm] and fixed, where Rm is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for computing the optimal or near-optimal value of Rm based on the differential evolution algorithm. This method seeks the optimal Rm by minimizing the residual norm of the linear or nonlinear algebraic system that results from the ELM representation of the PDE solution and that corresponds to the ELM least squares solution to the system. It amounts to a pre-processing procedure that determines a near-optimal Rm, which can be used in ELM for solving linear or nonlinear PDEs. The presented method enables us to illuminate the characteristics of the optimal Rm for two types of ELM configurations: (i) Single-Rm-ELM, corresponding to the conventional ELM method in which a single Rm is used for generating the random coefficients in all the hidden layers, and (ii) Multi-Rm-ELM, corresponding to a modified ELM method in which multiple Rm constants are involved with each used for generating the random coefficients of a different hidden layer. We adopt the optimal Rm from this method and also incorporate other improvements into the ELM implementation. In particular, here we compute all the differential operators involving the output fields of the last hidden layer by a forward-mode auto-differentiation, as opposed to the reverse-mode auto-differentiation in a previous work. These improvements significantly reduce the network training time and enhance the ELM performance. We systematically compare the computational performance of the current improved ELM with that of the finite element method (FEM), both the classical second-order FEM and the high-order FEM with Lagrange elements of higher degrees, for solving a number of linear and nonlinear PDEs. It is shown that the current improved ELM far outperforms the classical FEM. Its computational performance is comparable to that of the high-order FEM for smaller problem sizes, and for larger problem sizes the ELM markedly outperforms the high-order FEM.

show abstract

mentioning

confidence: 99%

On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

Dong,

Yang

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The main feature is that the input weight of the neuron in input layer to the neuron in hidden layer and the threshold of neuron in hidden layer can be randomly given and do not need to be adjusted, and the learning process only needs to calculate the output weight [47][48][49]. Therefore, ELM learns faster than traditional artificial neural networks while ensuring learning accuracy [50]. The structure of ELM is presented in Figure 4.…”

Section: Extreme Learning Machinementioning

confidence: 99%

Proton Exchange Membrane Fuel Cell Prognostics Using Genetic Algorithm and Extreme Learning Machine

2020

View full text Add to dashboard Cite

The prognostics can predict the degradation of proton exchange membrane fuel cell (PEMFC) to formulate a reasonable maintenance plan for improving its lifetime and performance. In this paper, the voltage degradation for PEMFC at different conditions is predicted, by using a novel prognostics method based on genetic algorithm (GA) and extreme learning machine (ELM). The novel prognostics method considers the effects of the PEMFC load current, relative humidity, hydrogen pressure, and temperature on the degradation for PEMFC. Firstly, the voltage degradation prediction model for PEMFC is built by the ELM. Then, the GA is adopted to determine the optimal parameter of the proposed degradation prediction model. Finally, the voltage degradation prediction of the proposed prognostics method is validated, using data derived from the PEMFC in actual postal fuel cell electric vehicle (PFCEV) at real conditions and PEMFC at dynamic load current. The experimental results show that the proposed prognostics method can obtain good voltage degradation prediction for PEMFC in PFCEV at real conditions. The proposed method achieves better voltage degradation prediction for PEMFC at dynamic load current than other traditional data‐based prognostics methods.

show abstract

“…Distributed smart space orchestration system (Ds2os) was the second IoT-related benchmark and included a collection of traces captured in a networking domain for IoT. 1 The data had been collected from the application layer; hence, they differed significantly from the conventional feature-based patterns used by network-traffic classifiers. The main dataset included various sources, such as light controllers, thermometers, person detection sensors, washing machines, batteries, thermostats, smart doors, and smart phones.…”

Section: Internet Of Things Benchmarksmentioning

confidence: 99%

“…First, the proposed designs mostly rely on reconfigurable platforms such as field programmable gate arrays (FPGAs) [9,10,37,50], which may prove quite expensive. By contrast, implementations on micro-controllers or microcomputers have drawn limited attention, in spite of the fact that these devices best fit IoT applications and remarkably shrink the time-to-market of commercial products [1,17].…”

Section: Introductionmentioning

confidence: 99%

Random-based networks with dropout for embedded systems

Ragusa

Gianoglio

Zunino

et al. 2020

Neural Comput & Applic

View full text Add to dashboard Cite

Random-based learning paradigms exhibit efficient training algorithms and remarkable generalization performances. However, the computational cost of the training procedure scales with the cube of the number of hidden neurons. The paper presents a novel training procedure for random-based neural networks, which combines ensemble techniques and dropout regularization. This limits the computational complexity of the training phase without affecting classification performance significantly; the method best fits Internet of Things (IoT) applications. In the training algorithm, one first generates a pool of random neurons; then, an ensemble of independent sub-networks (each including a fraction of the original pool) is trained; finally, the sub-networks are integrated into one classifier. The experimental validation compared the proposed approach with state-of-the-art solutions, by taking into account both generalization performance and computational complexity. To verify the effectiveness in IoT applications, the training procedures were deployed on a pair of commercially available embedded devices. The results showed that the proposed approach overall improved accuracy, with a minor degradation in performance in a few cases. When considering embedded implementations as compared with conventional architectures, the speedup of the proposed method scored up to 20× in IoT devices.

show abstract

Towards a more efficient and cost-sensitive extreme learning machine: A state-of-the-art review of recent trend

Cited by 51 publications

References 152 publications

On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

Proton Exchange Membrane Fuel Cell Prognostics Using Genetic Algorithm and Extreme Learning Machine

Random-based networks with dropout for embedded systems

Contact Info

Product

Resources

About