An on-line weighted ensemble of regressor models to handle concept drifts

Abstract

a b s t r a c tMany estimation, prediction, and learning applications have a dynamic nature. One of the most important challenges in machine learning is dealing with concept changes. Underlying changes may make the model designed on old data, inconsistent with new data. Also, algorithms usually specialize in one type of change. Other challenge is reusing previously acquired information in scenarios where changes may recur. This strategy improves the learning accuracy and reduces the processing time. Unfortunately, most existing learning algorithms to deal with changes are adapted on a batch basis. This process usually requires a long time, and such data may not reflect the current state of the system. However, even the system is adapted on a sample basis, existing algorithms may adapt slowly to changes and cannot conciliate old and new information. This paper proposes an On-line Weighted Ensemble (OWE) of regressor models which is able to learn incrementally sample by sample in the presence of several types of changes and simultaneously retain old information in recurring scenarios. The key idea is to keep a moving window that slides when a new sample is available. The error of each model on the current window is determined using a boosting strategy that assigns small errors to the models that predict accurately the samples predicted poorly by the ensemble. To handle recurring and non-recurring changes, OWE uses a new assignment of models' weights that takes into account the models' errors on the past and current windows using a discounting factor that decreases or increases the contribution of old windows. In addition, OWE launches new models if the system's accuracy is decreasing, and it can exclude inaccurate models over time. Experiments with artificial and industrial data reveal that in most cases OWE outperforms other state-of-the-art concept drift approaches.

Machine Learning in Predictive Maintenance towards Sustainable Smart Manufacturing in Industry 4.0

“…ANNs are primarily appropriate for systems with a complex, large scale structure and unclear information [4]. ANNs are widely applied and they are the most common ML algorithms [1], at the same time they have been suggested in several industrial applications involving soft sensing [44], and in predictive control systems [45]. Hesser, D.F.…”

Multi‐output regression for imbalanced data stream

“…In Soares and Araújo (2015), authors proposed an On‐line Weighted Ensemble (OWE) which is a set of single output regressor models in the non‐stationary data stream and uses a cost‐sensitive boosting strategy to assign small errors to the models that predict accurately the samples predicted poorly by the ensemble.…”

An adaptive online learning approach for Support Vector Regression: Online-SVR-FID

“…A total of M = 1000 data points are generated for each dataset. The fourth drifting dataset (Hyperplane) is generated similarly to [29], which includes four different concepts and generates totally M (=1000) data points; the dataset includes nine input variables that are uniformly distributed over the interval [0,1]. For these four artificial datasets, a noise generated by a Gaussian distribution with zero mean is added to the output and each input variable.…”