EnsPKDE&IncLKDE: a hybrid time series prediction algorithm integrating dynamic ensemble pruning, incremental learning, and kernel density estimation

Abstract

Ensemble pruning can effectively overcome several shortcomings of the classical ensemble learning paradigm, such as the relatively high time and space complexity. However, each predictor has its own unique ability. One predictor may not perform well on some samples, but it will perform very well on other samples. Blindly underestimating the power of specific predictors is unreasonable. Choosing the best predictor set for each query sample is exactly what dynamic ensemble pruning techniques address. This paper proposes a hybrid Time Series Prediction (TSP) algorithm to implement one-step-ahead prediction task, integrating Dynamic Ensemble Pruning (DEP), Incremental Learning (IL), and Kernel Density Estimation (KDE), abbreviated as the EnsP KDE &IncL KDE algorithm. It dynamically selects proper predictor sets based on the kernel density distribution of all base learners' prediction values. Due to the characteristic of TSP problems that samples arrive in chronological order, the idea of IL is naturally introduced into EnsP KDE &IncL KDE , while DEP is a common technology to address the concept drift issue inherent in IL. The algorithm is divided into three subprocesses: 1) Overproduction, which generates the original ensemble learning system; 2) Dynamic Ensemble Pruning (DEP), achieved by one subalgorithm called EnsP KDE ; 3) Incremental Learning (IL), realized by one subalgorithm termed IncL KDE . Benefited from the advantages of integrating Dynamic Ensemble Pruning scheme, Incremental Learning paradigm and Kernel Density Estimation, in the experimental results, EnsP KDE &IncL KDE demonstrates superior prediction performance to several other state-of-the-art algorithms in fulfilling time series forecasting tasks.

A Dynamic Predictor Selection Method Based on Recent Temporal Windows for Time Series Forecasting

“…Content may change prior to final publication. f orecasting = f orecasting ∪ {ẑ i t+1 } 15 end 16 return f orecasting dows, composed of a maximum of 20 lags [42], where these lags are selected using the Auto-correlation function (ACF) [43]. Each time series was split into three sequential samples with the following proportion: 50% for training, 25% for validation, and the last 25% observations for testing.…”

“…Dynamic selection approaches commonly use the performance of the models in the Region of Competence (RoC) as a criterion to select the most competent ones [9], [12], [13], [17]- [19]. The RoC is composed of the k patterns in the in-sample (training or validation sets) [12], which are more similar to the test pattern according to some measure such as the Euclidean distance [20].…”

Ensemble learning based on relative accuracy approach and diversity teams

“…Some works focused on the types of trainers, either homogenous [3], [4] or heterogeneous [5], [6]. Some works looked at the purpose of using ensemble learning either for classifying [7]- [26], clustering [27]- [34], regression [35]- [37], or streaming [38]- [42].…”