A demand for predictive models for on-line estimation of variables is increasing in industry. As industrial processes are timevarying, on-line learning algorithms should be adaptive to capture process changes. On-line ensemble methods have been shown to provide better generalization performance than single models in changing environments. However, most on-line ensembles do not include and exclude models during on-line operation. As a result, the ensembles have limited adaptation capability. Moreover, a higher performance can be obtained by combining a selected set of most relevant models of the ensemble for the current situation, rather than combining all the models. This paper proposes a new on-line learning ensemble of regressor models using an ordered aggregation (OA) technique which is able to provide on-line predictions of variables in changing environments. OA dynamically selects an optimal size and composition of a subset of models based on the minimization of the ensemble error on the newest sample. The proposed strategy overcomes the problem of defining the optimal ensemble size, and in most cases it obtains better performance than aggregating all the models. Models are added or removed for assuring adaptation of the ensemble in changing environments. Furthermore, this paper proposes and integrates a new on-line Extreme Learning Machine (ELM) neural network model with variable forgetting factor (FF) using the directional FF method which shows superior performance in industrial applications when compared to the well-known On-line Sequential ELM (OS-ELM) algorithm. Experiments are reported to demonstrate the performance and effectiveness of the proposed methods.
In the last decades ensemble learning has established itself as a valuable strategy within the computational intelligence modeling and machine learning community. Ensemble learning is a paradigm where multiple models combine in some way their decisions, or their learning algorithms, or different data to improve the prediction performance. Ensemble learning aims at improving the generalization ability and the reliability of the system. Key factors of ensemble systems are diversity, training and combining ensemble members to improve the ensemble system performance. Since there is no unified procedure to address all these issues, this work proposes and compares Genetic Algorithm and Simulated Annealing based approaches for the automatic development of Neural Network Ensembles for regression problems. The main contribution of this work is the development of optimization techniques that select the best subset of models to be aggregated taking into account all the key factors of ensemble systems (e.g., diversity, training ensemble members and combination strategy). Experiments on two well-known data sets are reported to evaluate the effectiveness of the proposed methodologies. Results show that these outperform other approaches including Simple Bagging, Negative Correlation Learning (NCL), AdaBoost and GASEN in terms of generalization ability.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.