Online Sequential Extreme Learning Machine With Kernels

Abstract

The extreme learning machine (ELM) was recently proposed as a unifying framework for different families of learning algorithms. The classical ELM model consists of a linear combination of a fixed number of nonlinear expansions of the input vector. Learning in ELM is hence equivalent to finding the optimal weights that minimize the error on a dataset. The update works in batch mode, either with explicit feature mappings or with implicit mappings defined by kernels. Although an online version has been proposed for the former, no work has been done up to this point for the latter, and whether an efficient learning algorithm for online kernel-based ELM exists remains an open problem. By explicating some connections between nonlinear adaptive filtering and ELM theory, in this brief, we present an algorithm for this task. In particular, we propose a straightforward extension of the well-known kernel recursive least-squares, belonging to the kernel adaptive filtering (KAF) family, to the ELM framework. We call the resulting algorithm the kernel online sequential ELM (KOS-ELM). Moreover, we consider two different criteria used in the KAF field to obtain sparse filters and extend them to our context. We show that KOS-ELM, with their integration, can result in a highly efficient algorithm, both in terms of obtained generalization error and training time. Empirical evaluations demonstrate interesting results on some benchmarking datasets.

Electric-Parameter-Based Inversion of Dynamometer Card Using Hybrid Modeling for Beam Pumping System

“…In order to approve the effectiveness of the proposed hybrid prediction method, the mechanism predictive model without data-driven model and pure IFS-DDKOS-ELM model are chosen to compare the prediction performance, and the prediction performances are shown as Figure 11(a). In addition, combination kernel function LSSVM and the KOS-ELM in [25] are selected as the comparison algorithms to verify the effectiveness of the IFS-DDKOS-ELM. The prediction effect of the KOS-ELM in [25] is proved to be better than ELM, K-ELM, and OS-ELM.…”

“…In addition, combination kernel function LSSVM and the KOS-ELM in [25] are selected as the comparison algorithms to verify the effectiveness of the IFS-DDKOS-ELM. The prediction effect of the KOS-ELM in [25] is proved to be better than ELM, K-ELM, and OS-ELM. So this paper directly chooses these two methods to compare the prediction performance of the IFS-DDKOS-ELM model.…”

“…Therefore, in order to ease the computation complexity of the inverse matrix in (18) and extend the kernel extreme learning machine to the online version, Simone et al [25] proposed a kernel online sequential extreme learning machine(KOS-ELM) by straightforwardly extending the kernel recursive least squares to the online sequential extreme learning machine(OS-ELM) framework.…”

Multi-label Learning Based on Kernel Extreme Learning Machine

“…Regarding positive definiteness of kernel ELM output weight matrix, Cholesky decomposition [3] is adopted to solve hidden layer output matrix inversion problem. Cholesky method is successfully applied to time-varying or nonstationary systems, while incremental updating of the system is implemented [4][5][6]. Inspired by this method, kernel extreme learning machine is applied to multi-label classification problem, which is called KELM-ML.…”

Prediction of Burden Descent Speed Based on Improved Extreme Learning Machine

“…Extreme Learning Machine (ELM) algorithm is proposed in order to overcome the issues of the conventional gradient-based learning algorithms for SLFNs (Huang and Zhu,2006;Scardapane and Comminiello, 2015). Compared with other neural network algorithms, ELM has more fast training speed.…”