Improving nonparametric regression methods by bagging and boosting

Borra, Simone; Ciaccio, Agostino Di

doi:10.1016/s0167-9473(01)00068-8

Cited by 54 publications

(27 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The fundamental reason for the effectiveness of bagging unstable models is that the predictive capability of the regression models is based on the bias/variance trade-off, and the variance of the aggregated predictor is reduced and close-to-constant bias is maintained [52].…”

Section: Bagging and Subagging For Non-linear Multivariate Calibrationmentioning

confidence: 99%

Bagging for robust non-linear multivariate calibration of spectroscopy

Wang

Chen

Lau

2011

Chemometrics and Intelligent Laboratory Systems

View full text Add to dashboard Cite

Section: Bagging and Subagging For Non-linear Multivariate Calibrationmentioning

confidence: 99%

Bagging for robust non-linear multivariate calibration of spectroscopy

Wang

Chen

Lau

2011

Chemometrics and Intelligent Laboratory Systems

View full text Add to dashboard Cite

“…It has been observed that this simple averaging rule can significantly improve the model accuracy and robustness in various applications [4,6,22,23].…”

Section: Combination Using Simple Averaging Rulementioning

confidence: 99%

Bagging for Gaussian process regression

2009

View full text Add to dashboard Cite

This paper proposes the application of bagging to obtain more robust and accurate predictions using Gaussian process regression models. The training data is re-sampled using the bootstrap method to form several training sets, from which multiple Gaussian process models are developed and combined through weighting to provide predictions. A number of weighting methods for model combination are discussed, including the simple averaging rule and the weighted averaging rules. We propose to weight the models by the inverse of their predictive variance, and thus the prediction uncertainty of the models is automatically accounted for. The bagging method for Gaussian process regression is successfully applied to the inferential estimation of quality variables in an industrial chemical plant.

show abstract

“…[3,5,17,26]. It has, however, also been successfully applied to regressions such as projection pursuit regression (PPR), multivariate adaptive regression splines (MARS), local learning based on recursive covering (DART), support vector machines (SVM), regression trees and and linear regression with forward variable selection [2,4,5,24].…”

Section: Baggingmentioning

confidence: 99%

“…They can lead to increased accuracy of the predictors and also better stability. In particular, ensemble methods have good properties when used in connection with regression and classification trees [3,5,16], but they have also given promising results for other methods [4,5,17,24,26].…”

Section: Introductionmentioning

confidence: 99%

Ensemble methods and partial least squares regression

Mevik

Segtnan

Næs

2004

Journal of Chemometrics

View full text Add to dashboard Cite

Recently, there has been an increased attention in the literature on the use of ensemble methods in multivariate regression and classification. These methods have been shown to have interesting properties both for regression and classification. In particular, they can improve the accuracy of unstable predictors.Ensemble methods have so far, been little studied in situations that are common for calibration and prediction in chemistry, i.e., situations with a large number of collinear x-variables and few samples. These situations are often approached by data compression methods such as principal components regression (PCR) or partial least squares regression (PLSR).The present paper is an investigation of the properties of different types of ensemble methods used with PLSR in situations with highly collinear x-data. Bagging and data augmentation by simulated noise are studied. The focus is on the robustness of the calibrations. Real and simulated data is used.The results show that ensembles trained on data with added noise can make the PLSR robust against the type of noise added. In particular, the effects of sample temperature variations can be eliminated. Bagging does not seem to give any improvement over PLSR for small and intermediate number of components. It is, however, less sensitive to overfitting.

show abstract

Improving nonparametric regression methods by bagging and boosting

Cited by 54 publications

References 10 publications

Bagging for robust non-linear multivariate calibration of spectroscopy

Bagging for robust non-linear multivariate calibration of spectroscopy

Bagging for Gaussian process regression

Ensemble methods and partial least squares regression

Contact Info

Product

Resources

About