Dynamic and Heterogeneous Ensembles for Time Series Forecasting

Abstract

This paper addresses the issue of learning time series forecasting models in changing environments by leveraging the predictive power of ensemble methods. Concept drift adaptation is performed in an active manner, by dynamically combining base learners according to their recent performance using a non-linear function. Diversity in the ensembles is encouraged with several strategies that include heterogeneity among learners, sampling techniques and computation of summary statistics as extra predictors. Heterogeneity is used with the goal of better coping with different dynamic regimes of the time series. The driving hypotheses of this work are that (i) heterogeneous ensembles should better fit different dynamic regimes and (ii) dynamic aggregation should allow for fast detection and adaptation to regime changes. We extend some strategies typically used in classification tasks to time series forecasting. The proposed methods are validated using Monte Carlo simulations on 16 realworld univariate time series with numerical outcome as well as an artificial series with clear regime shifts. The results provide strong empirical evidence for our hypotheses. To encourage reproducibility the proposed method is publicly available as a software package.

Prediction of liquid ammonia yield using a novel deep learning‐based heterogeneous pruning ensemble model

“…12,13 Despite the popular practice of using identical type of models in an "ensemble" to handle chemical engineering problems, [14][15][16] the combination of models with essentially distinct structures (defined as "heterogeneous" here), seems to be more attractive. 17,18 As pointed out by Vinay et al, 19 the heterogeneous ensemble significantly improves model performance with regard to generalization ability. Therefore, in this investigation, nine base models mentioned in Ge et al 10 form a "pool" to be considered for introduction to the ensemble.…”

“…Recent literature reports from different fields confirm that multi‐model ensemble could combine the advantages of base models and lead to better predictive performance . Despite the popular practice of using identical type of models in an “ensemble” to handle chemical engineering problems, the combination of models with essentially distinct structures (defined as “heterogeneous” here), seems to be more attractive . As pointed out by Vinay et al, the heterogeneous ensemble significantly improves model performance with regard to generalization ability.…”

A study on Ensemble Learning for Time Series Forecasting and the need for Meta-Learning

“…7) Ensemble based on performance of most recent data points (recent ensemble): This technique, adapted from [24], selects and combines base models based on the performance on the most recent data points from the training dataset. The technique selects the best-performing λ% models and computes their weights by taking into account their performance on the P most recent points of the train dataset.…”

“…The parameters λ and P are therefore hyperparameters. Contrary to [24], we use the sAPE (sMAPE without mean) instead of the Squared Error to compute the performance score for scaling reasons, and do not update our train set over time.…”

Model Compression for Dynamic Forecast Combination

“…The idea behind this approach is that the short-term future will be similar to recent past and earlier observations are not as relevant. Windowing approaches have been used to weight and combine the available models (Newbold & Granger, 1974;Cerqueira et al, 2017) (WL), or to select the recent best performing one (van Rijn et al, 2015) (BLAST).…”