Two-hidden-layer extreme learning machine for regression and classification

Qu, Boyang; Lang, Bingbing; Liang, Jing; Qin, A. K.; Crisalle, O.D.

doi:10.1016/j.neucom.2015.11.009

Cited by 96 publications

(40 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Now the TELM defines the matrix = [ 1 1 ], so the parameters of the second hidden layer can be easily obtained by using formula (12) and the inverse function of the activation function.…”

Section: Two-hidden-layer Elmmentioning

confidence: 99%

“…But ELM only needs to randomly set the weights and bias of the hidden neurons, and the output weights are determined by using the Moore-Penrose pseudoinverse under the criterion of least-squares method. In recent years, various ELM variants have been proposed aiming to achieve better achievements, such as the deep ELM with kernel based on Multilayer Extreme Learning Machine (DELM) algorithm [11]; two-hidden-layer extreme learning machine (TELM) [12]; a Four-Layered Feedforward Neural Network [13]; online sequential extreme learning machine [14,15]; multiple kernel extreme learning machine (MK-ELM) [16]; two-stage extreme learning machine [17], using noise detection and improving the classifier accuracy [18,19].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Multiple Hidden Layers Extreme Learning Machine Method and Its Application

Xiao

Mao

2017

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Extreme learning machine (ELM) is a rapid learning algorithm of the single-hidden-layer feedforward neural network, which randomly initializes the weights between the input layer and the hidden layer and the bias of hidden layer neurons and finally uses the least-squares method to calculate the weights between the hidden layer and the output layer. This paper proposes a multiple hidden layers ELM (MELM for short) which inherits the characteristics of parameters of the first hidden layer. The parameters of the remaining hidden layers are obtained by introducing a method (make the actual output zero error approach the expected hidden layer output). Based on the MELM algorithm, many experiments on regression and classification show that the MELM can achieve the satisfactory results based on average precision and good generalization performance compared to the two-hidden-layer ELM (TELM), the ELM, and some other multilayer ELM.

show abstract

“…Now the TELM defines the matrix = [ 1 1 ], so the parameters of the second hidden layer can be easily obtained by using formula (12) and the inverse function of the activation function.…”

Section: Two-hidden-layer Elmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Multiple Hidden Layers Extreme Learning Machine Method and Its Application

Xiao

Mao

2017

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…In order to improve the embarrassing situation of RELM, the literature [17,18] starts from improving its network structure. On the basis of the traditional RELM threelayer structure, the number of hidden layers is increased to form a neural network with one input layer, multiple hidden layers, and one output layer, that is, the multiple hidden layers RELM network model (MRELM), in which the neuron nodes of each hidden layer are fully connected.…”

Section: Introductionmentioning

confidence: 99%

Incremental Multiple Hidden Layers Regularized Extreme Learning Machine Based on Forced Positive‐Definite Cholesky Factorization

Liu

2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The theory and implementation of extreme learning machine (ELM) prove that it is a simple, efficient, and accurate machine learning method. Compared with other single hidden layer feedforward neural network algorithms, ELM is characterized by simpler parameter selection rules, faster convergence speed, and less human intervention. The multiple hidden layer regularized extreme learning machine (MRELM) inherits these advantages of ELM and has higher prediction accuracy. In the MRELM model, the number of hidden layers is randomly initiated and fixed, and there is no iterative tuning process. However, the optimal number of hidden layers is the key factor to determine the generalization ability of MRELM. Given this situation, it is obviously unreasonable to determine this number by trial and random initialization. In this paper, an incremental MRELM training algorithm (FC-IMRELM) based on forced positive-definite Cholesky factorization is put forward to solve the network structure design problem of MRELM. First, an MRELM-based prediction model with one hidden layer is constructed, and then a new hidden layer is added to the prediction model in each training step until the generalization performance of the prediction model reaches its peak value. Thus, the optimal network structure of the prediction model is determined. In the training procedure, forced positive-definite Cholesky factorization is used to calculate the output weights of MRELM, which avoids the calculation of the inverse matrix and Moore-Penrose generalized inverse of matrix involved in the training process of hidden layer parameters. Therefore, FC-IMRELM prediction model can effectively reduce the computational cost brought by the process of increasing the number of hidden layers. Experiments on classification and regression problems indicate that the algorithm can be effectively used to determine the optimal network structure of MRELM, and the prediction model training by the algorithm has excellent performance in prediction accuracy and computational cost.

show abstract

“…This approach maintains testing accuracy with stable condition, but it was only limited to ELM with linear hidden neurons. In recent years, many studies have focused on the structural improvements of ELM, such as ELM with double hidden layers [14] and ELM with multiple hidden layers [15,16]. The spectral data of haematite is a set of highly coupled and nonlinear matrices.…”

Section: Introductionmentioning

confidence: 99%

Study of the Magnetic Properties of Haematite Based on Spectroscopy and the IPSO-ELM Neural Network

et al. 2018

View full text Add to dashboard Cite

The detection of the magnetic properties of haematite plays an important role in the adjustment of the beneficiation process of haematite and the improvement of metal recovery. The existing methods for measuring the magnetic properties of iron ore either have large errors or take a long time. Therefore, it is very necessary to find a method that can quickly and accurately detect the magnetic properties of haematite. This paper presents a method to detect the magnetic properties of haematite based on the extreme learning machine based on the improved particle swarm optimization (IPSO-ELM) algorithm and spectroscopy. The improved particle swarm optimization algorithm is used to optimize the input weights, hidden layer deviations, and hidden layer nodes of the ELM network. Introducing the linear decreasing inertia weight for the particle swarm algorithm, taking into account the norm of the output weight in the particle update process and using the variation idea to change the length of the particle give the IPSO-ELM better stability and generalization ability. The experimental results show that the IPSO-ELM prediction model has a good prediction performance and has better generalization ability than that of the ELM and PSO-ELM prediction models. Compared with traditional chemical analysis methods and manual methods, this method has great advantages in terms of economy, speed, and accuracy.

show abstract

Two-hidden-layer extreme learning machine for regression and classification

Cited by 96 publications

References 28 publications

A Multiple Hidden Layers Extreme Learning Machine Method and Its Application

A Multiple Hidden Layers Extreme Learning Machine Method and Its Application

Incremental Multiple Hidden Layers Regularized Extreme Learning Machine Based on Forced Positive‐Definite Cholesky Factorization

Study of the Magnetic Properties of Haematite Based on Spectroscopy and the IPSO-ELM Neural Network

Contact Info

Product

Resources

About