Fast kernel extreme learning machine for ordinal regression

Shi, Yong; Li, Peijia; Yuan, Hao; Miao, Jianyu; Niu, Lingfeng

doi:10.1016/j.knosys.2019.04.003

Cited by 17 publications

(17 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zaher et al [47] implemented ELM in predicting compressive strength of lightweight foamed concrete. Yong Shi et al [48] recommended fast learning machine for ordinal regression. Ahmed et al [49] used ELM model in water network management.…”

Section: Fig 16: Types Of Regression In Machine Learningmentioning

confidence: 99%

WITHDRAWN: An Analysis of Extreme Machine Learning Models

Pande¹,

Rathore²,

Purohit³

2021

Preprint

View full text Add to dashboard Cite

Machine learning applications employ FFNN (Feed Forward Neural Network) in their discipline enormously. But, it has been observed that the FFNN requisite speed is not up the mark. The fundamental causes of this problem are: 1) for training neural networks, slow gradient descent methods are broadly used and 2) for such methods, there is a need for iteratively tuning hidden layer parameters including biases and weights. To resolve these problems, a new emanant machine learning algorithm, which is a substitution of the feed-forward neural network, entitled as Extreme Learning Machine (ELM) introduced in this paper. ELM also come up with a general learning scheme for the immense diversity of different networks (SLFNs and multilayer networks). According to ELM originators, the learning capacity of networks trained using backpropagation is a thousand times slower than the networks trained using ELM, along with this, ELM models exhibit good generalization performance. ELM is more efficient in contradiction of Least Square Support Vector Machine (LS-SVM), Support Vector Machine (SVM), and rest of the precocious approaches. ELM’s eccentric outline has three main targets: 1) high learning accuracy 2) less human intervention 3) fast learning speed. ELM consider as a greater capacity to achieve global optimum. The distribution of application of ELM incorporates: feature learning, clustering, regression, compression, and classification. With this paper, our goal is to familiarize various ELM variants, their applications, ELM strengths, ELM researches and comparison with other learning algorithms, and many more concepts related to ELM.

show abstract

Section: Fig 16: Types Of Regression In Machine Learningmentioning

confidence: 99%

WITHDRAWN: An Analysis of Extreme Machine Learning Models

Pande¹,

Rathore²,

Purohit³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In our paper, we build the ordinal regression by using this reduction framework and SVM. Recently, [22] presented a new Kernel Extreme Learning Machine for ordinal regression (KELMOR) by exploiting a quadratic cost-sensitive encoding scheme to deal with the efficiency of OR in the big data scenario. [23] proposed a novel ordinal regression model , which is named as nonparallel support vector ordinal regression (NPSVOR), a set of possible nonparallel hyperplanes are constructed independently.…”

Section: A Ordinal Regressionmentioning

confidence: 99%

Active Learning for Imbalanced Ordinal Regression

Chen

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Ordinal regression (OR), also called ordinal classification, is a special multi-classification designed for problems with ordered classes. Imbalanced data hinders the performance of classification algorithms, especially for OR algorithms, as imbalanced class distributions often arise in OR problems. In this paper, we address an active learning based solution for imbalanced OR problem. We propose an active learning algorithm for OR (AL-OR) to select the most informative samples from unlabeled samples, mark them and add them to the training set. Based on AL-OR, we put forward an improved active learning for imbalanced OR (IAL-IOR), which further adjust the sampling strategy of AL-OR dynamically to make the training data as valuable and balanced as possible. Recall rate for multi-classification and new mean absolute error are designed to measure the performance of the algorithms. To the best of our knowledge, our algorithm is the first algorithm for imbalanced OR in algorithm level. The experimental results show that the proposed algorithms have faster convergence and much better generalization ability than the classical methods and the state-of-the-art methods under the evaluation measurements for imbalance problems. In addition, we also proved the effectiveness of our algorithms by statistical analysis.

show abstract

“…In the ELM algorithm, the input weights and hidden biases are randomly generated from any continuous probability distribution, and then, the output weights can be solved using the generalized Moore-Penrose inverse. Compared with the BP neural network, this algorithm has a good performance network in regression [4][5][6], classification [7][8][9], feature learning [10][11][12], and cluster tasks [13][14][15]. Different from conventional gradient-based neural network learning algorithms, which are sensitive to the combination of parameters and easy to trap in local optimum, ELM not only has a faster training speed but also has a smaller training error.…”

Section: Introductionmentioning

confidence: 99%

Pruning Multilayered ELM Using Cholesky Factorization and Givens Rotation Transformation

Liu

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Extreme learning machine is originally proposed for the learning of the single hidden layer feedforward neural network to overcome the challenges faced by the backpropagation (BP) learning algorithm and its variants. Recent studies show that ELM can be extended to the multilayered feedforward neural network in which the hidden node could be a subnetwork of nodes or a combination of other hidden nodes. Although the ELM algorithm with multiple hidden layers shows stronger nonlinear expression ability and stability in both theoretical and experimental results than the ELM algorithm with the single hidden layer, with the deepening of the network structure, the problem of parameter optimization is also highlighted, which usually requires more time for model selection and increases the computational complexity. This paper uses Cholesky factorization strategy and Givens rotation transformation to choose the hidden nodes of MELM and obtains the number of nodes more suitable for the network. First, the initial network has a large number of hidden nodes and then uses the idea of ridge regression to prune the nodes. Finally, a complete neural network can be obtained. Therefore, the ELM algorithm eliminates the need to manually set nodes and achieves complete automation. By using information from the previous generation’s connection weight matrix, it can be evitable to re-calculate the weight matrix in the network simplification process. As in the matrix factorization methods, the Cholesky factorization factor is calculated by Givens rotation transform to achieve the fast decreasing update of the current connection weight matrix, thus ensuring the numerical stability and high efficiency of the pruning process. Empirical studies on several commonly used classification benchmark problems and the real datasets collected from coal industry show that compared with the traditional ELM algorithm, the pruning multilayered ELM algorithm proposed in this paper can find the optimal number of hidden nodes automatically and has better generalization performance.

show abstract

Fast kernel extreme learning machine for ordinal regression

Cited by 17 publications

References 35 publications

WITHDRAWN: An Analysis of Extreme Machine Learning Models

WITHDRAWN: An Analysis of Extreme Machine Learning Models

Active Learning for Imbalanced Ordinal Regression

Pruning Multilayered ELM Using Cholesky Factorization and Givens Rotation Transformation

Contact Info

Product

Resources

About