Experiments with AdaBoost.RT, an Improved Boosting Scheme for Regression

Shrestha, Durga Lal; Solomatine, Dimitri

doi:10.1162/neco.2006.18.7.1678

Cited by 189 publications

(116 citation statements)

References 18 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…Hastie et al (2009) apply it to classi…cation, and argue that "boosting" is one of the most powerful learning algorithms currently available. The method has been extended to regression problems in Ridgeway et al (1999) and Shrestha and Solomatine (2006). In the economics literature, Bai and Ng (2009) use a boosting for selecting the predictors in factor augmented autoregressions.…”

Section: Robust Estimation Techniquesmentioning

confidence: 99%

Mining Big Data Using Parsimonious Factor and Shrinkage Methods

Kim

Swanson

2013

SSRN Journal

View full text Add to dashboard Cite

A number of recent studies have focused on the usefulness of factor models in the context of prediction using "big data" (see e.g., Bai and Ng (2008), Dufour and Stevanovic (2010), Forni et al. (2000, Kim and Swanson (2014), Stock and Watson (2002b, and the references cited therein). We add to this literature by analyzing the predictive bene…ts associated with the use of independent component analysis (ICA) and sparse principal component analysis (SPCA), coupled with a variety of other factor estimation and data shrinkage methods, including bagging, boosting, and the elastic net, among others. We carry out a forecasting "horse-race", involving the estimation of 28 di¤erent baseline model types, each constructed using a variety of speci…cation approaches, estimation approaches, and benchmark econometric models; and all used in the prediction of 11 key macroeconomic variables relevant for monetary policy assessment. In numerous instances we …nd that a variety of benchmark autoregressive models and model averaging methods are mean square forecast error (MSFE) dominated by more complicated nonlinear methods. For example, simple averaging methods are MSFE "best" in only 9 of 33 key cases considered. However, in order to "beat" model averaging methods, we must combine new factor estimation methods with interesting new forms of shrinkage. For example, SPCA yields MSFE-best prediction models in many cases, particularly when coupled with shrinkage. In summary, we present empirical results that provide strong new evidence of the usefulness of sophisticated factor based forecasting methods, and as a corollary, of the use of "big data" in macroeconometric forecasting.

show abstract

Section: Robust Estimation Techniquesmentioning

confidence: 99%

Mining Big Data Using Parsimonious Factor and Shrinkage Methods

Kim

Swanson

2013

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…The Adaboost paradigm has been also considered for regression problem. In [22] the authors present two algorithms denoted AdaBoost.R2 and AdaBoost.RT. In AdaBoost.R2, all weights are updated at each iteration while in AdaBoost.RT the weights are updated according to some threshold, in such a way to emphasis on the only examples hard to classify.…”

Section: Adaboostmentioning

confidence: 99%

Age Regression Based on Local Image Features

Mokadem

Charbit

Chollet

et al. 2010

2010 Fourth Pacific-Rim Symposium on Image and Video Technology

View full text Add to dashboard Cite

“…This is accomplished by finding a linear combination of weak learning algorithms in order to minimize the total loss over a set of training data commonly using a functional gradient descent [19,20]. Boosting is successfully applied to several different problems in the machine learning literature including classification [1,20,21], regression [19,21,22], and prediction [23,24]. However, significantly less attention is given to the idea of boosting in online regression framework.…”

Section: Introductionmentioning

confidence: 99%

Boosted adaptive filters

Kari

Mirza

Khan

et al. 2018

Digital Signal Processing

View full text Add to dashboard Cite

We investigate boosted online regression and propose a novel family of regression algorithms with strong theoretical bounds. In addition, we implement several variants of the proposed generic algorithm. We specifically provide theoretical bounds for the performance of our proposed algorithms that hold in a strong mathematical sense. We achieve guaranteed performance improvement over the conventional online regression methods without any statistical assumptions on the desired data or feature vectors. We demonstrate an intrinsic relationship, in terms of boosting, between the adaptive mixture-of-experts and data reuse algorithms. Furthermore, we introduce a boosting algorithm based on random updates that is significantly faster than the conventional boosting methods and other variants of our proposed algorithms while achieving an enhanced performance gain. Hence, the random updates method is specifically applicable to the fast and high dimensional streaming data. Specifically, we investigate Recursive Least Squares (RLS)-based and Least Mean Squares (LMS)-based linear regression algorithms in a mixture-of-experts setting, and provide several variants of these well known adaptation methods. Moreover, we extend the proposed algorithms to other filters. Specifically, we investigate the effect of the proposed algorithms on piecewise linear filters. Furthermore, we provide theoretical bounds for the computational complexity of our proposed algorithms. We demonstrate substantial performance gains in terms of mean square error over the constituent filters through an extensive set of benchmark real data sets and simulated examples.

show abstract

Experiments with AdaBoost.RT, an Improved Boosting Scheme for Regression

Cited by 189 publications

References 18 publications

Mining Big Data Using Parsimonious Factor and Shrinkage Methods

Mining Big Data Using Parsimonious Factor and Shrinkage Methods

Age Regression Based on Local Image Features

Boosted adaptive filters

Contact Info

Product

Resources

About