An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction

Zhang, Wei; Xu, Aiqiang; Ping, Dianfa; Mingzhe, Gao

doi:10.1007/s00521-017-3096-3

Cited by 13 publications

(7 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They can learn data one by one or piece by piece. Besides, unlike the former two algorithms, the proposed EnsP KDE &IncL KDE algorithm is designed based upon the dynamic ensemble learning paradigm, and a novel frame of incremental learning is constructed and incorporated into EnsP KDE &IncL KDE , therefore, two incremental learning algorithms, i.e., Kernel based Incremental ELM (KB-IELM) [34] and Online Incremental SVR (OI-SVR) [35], are also employed as the comparison algorithms. Finally, for comparison with deep learning algorithms, the Long Short Term Memory (LSTM) network [30] commonly used in TSP is utilized as another comparison algorithm.…”

Section: Methodsmentioning

confidence: 99%

EnsPKDE&IncLKDE: a hybrid time series prediction algorithm integrating dynamic ensemble pruning, incremental learning, and kernel density estimation

Zhu

Dai

2020

Appl Intell

View full text Add to dashboard Cite

Ensemble pruning can effectively overcome several shortcomings of the classical ensemble learning paradigm, such as the relatively high time and space complexity. However, each predictor has its own unique ability. One predictor may not perform well on some samples, but it will perform very well on other samples. Blindly underestimating the power of specific predictors is unreasonable. Choosing the best predictor set for each query sample is exactly what dynamic ensemble pruning techniques address. This paper proposes a hybrid Time Series Prediction (TSP) algorithm to implement one-step-ahead prediction task, integrating Dynamic Ensemble Pruning (DEP), Incremental Learning (IL), and Kernel Density Estimation (KDE), abbreviated as the EnsP KDE &IncL KDE algorithm. It dynamically selects proper predictor sets based on the kernel density distribution of all base learners' prediction values. Due to the characteristic of TSP problems that samples arrive in chronological order, the idea of IL is naturally introduced into EnsP KDE &IncL KDE , while DEP is a common technology to address the concept drift issue inherent in IL. The algorithm is divided into three subprocesses: 1) Overproduction, which generates the original ensemble learning system; 2) Dynamic Ensemble Pruning (DEP), achieved by one subalgorithm called EnsP KDE ; 3) Incremental Learning (IL), realized by one subalgorithm termed IncL KDE . Benefited from the advantages of integrating Dynamic Ensemble Pruning scheme, Incremental Learning paradigm and Kernel Density Estimation, in the experimental results, EnsP KDE &IncL KDE demonstrates superior prediction performance to several other state-of-the-art algorithms in fulfilling time series forecasting tasks.

show abstract

Section: Methodsmentioning

confidence: 99%

EnsPKDE&IncLKDE: a hybrid time series prediction algorithm integrating dynamic ensemble pruning, incremental learning, and kernel density estimation

Zhu

Dai

2020

Appl Intell

View full text Add to dashboard Cite

show abstract

“…The typical online classification algorithms include TSVDD [13] , eAdaBoost [14] , KB-IELM [15] , OS-ELM [16] , etc.…”

Section: The Related Research Work and Algorithmsmentioning

confidence: 99%

A Pruning Optimized Fast Learn++NSE Algorithm

et al. 2021

View full text Add to dashboard Cite

Due to the large number of typical applications, it is very important and urgent to study the fast classification learning of accumulated big data in nonstationary environments. The newly proposed algorithm, named Learn++.NSE, is one of the important research results in this research field. And a pruning version, named Learn++.NSE-Error-based, was given for accumulated big data to improve the learning efficiency. However, the studies have found that the Learn++.NSE-Error-based algorithm often encounters a situation that the newly generated base classifier is pruned in the next integration, which reduces the accuracy of the ensemble classifier. The newly generated base classifier is very important in the next ensemble learning and should be retained. Therefore, the two latest base classifiers are retained without being pruned, and a new pruning algorithm named NewLearn++.NSE-Error-based was proposed. The experimental results on the generated dataset and the real-world dataset show that NewLearn++.NSE-Error-based can further improve the accuracy of the ensemble classifier under the premise of obtaining the same time complexity as Learn++.NSE algorithm. It is suitable for fast classification learning of long-term accumulated big data.

show abstract

“…However, over time the distribution and changing trend of online data change [33], which put forward new model requirements. Zhang Wei et al [34] introduced an adaptive regularization factor to resolve the structural risk of the model in different nonlinear regions. Updating the kernel weight coefficient [35] is carried out additionally to improve the identification accuracy.…”

Section: Introductionmentioning

confidence: 99%

“…KB-IELM[23], (2)ALD-KOS-ELM[20], (3)NOS-KELM[34], (4)FF-OSKELM[22] (fixed forgetting factor which 01   ). Here OS-ELM methods are not considered for that the randomness of the initial weight setting of ELM input layer can lead to the randomness of the experimental results[19].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Online Sequential Model for Multivariate Time Series Prediction With Adaptive Forgetting Factor

Dai¹,

Xu²,

Liu³

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

In the process of online prediction of multivariable non-stationary time series by kernel extreme learning machine (KELM), the dynamic characteristics of the system which are difficult to determine have always posed a big problem. We propose an online sequential prediction model with an adaptive forgetting factor (AFF) for multivariable time series to solve this problem. The multivariable time series instead of variable itself is reconstructed firstly. AFF is introduced into the objective function and can be adjusted iteratively and adaptively with the system changes. As a result, higher weight can be allocated for the fresh and more important samples while the old failure samples can be quickly forgotten. The model sparsification uses a fast leave-one-out cross-validation (FLOO-CV) method to set a prediction error threshold so that samples can be selected conditionally to form a dictionary. Besides, the dictionary parameters, including AFF and kernel parameters, are recursively updated simultaneously without increasing calculation complexity. The experimental results show that, compared with four fashionable KLEM methods, the proposed AFF-OSKELM has a better dynamic tracking ability and adaptability. Moreover, compared with single variable prediction, the spatial reconstructed multivariable has higher prediction accuracy and stability. INDEX TERMS Kernel extreme learning machine, adaptive forgetting factor, fast leave-one-out crossvalidation, online prediction, multivariable time series.

show abstract

An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction

Cited by 13 publications

References 35 publications

EnsPKDE&IncLKDE: a hybrid time series prediction algorithm integrating dynamic ensemble pruning, incremental learning, and kernel density estimation

EnsPKDE&IncLKDE: a hybrid time series prediction algorithm integrating dynamic ensemble pruning, incremental learning, and kernel density estimation

A Pruning Optimized Fast Learn++NSE Algorithm

Online Sequential Model for Multivariate Time Series Prediction With Adaptive Forgetting Factor

Contact Info

Product

Resources

About