Approximation Stability and Boosting

Gao, Wei; Zhou, Zhi‐Hua

doi:10.1007/978-3-642-16108-7_9

Cited by 4 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use algorithmic stability to show that our collaborative approaches ensure good generalization. By extending findings in refs 48 , 49 , 50 , 51 , SI provides proofs that our collaborative strategies give lower and tighter upper bounds of the generalization error. Algorithmic stability has been utilized as reversed engineering: to guide which instances to exchange in order to improve the overall stability and reduce generalization error.…”

Section: Resultssupporting

confidence: 70%

Generating highly accurate prediction hypotheses through collaborative ensemble learning

Arsov

Pavlovski

Basnarkov

et al. 2017

Sci Rep

View full text Add to dashboard Cite

Ensemble generation is a natural and convenient way of achieving better generalization performance of learning algorithms by gathering their predictive capabilities. Here, we nurture the idea of ensemble-based learning by combining bagging and boosting for the purpose of binary classification. Since the former improves stability through variance reduction, while the latter ameliorates overfitting, the outcome of a multi-model that combines both strives toward a comprehensive net-balancing of the bias-variance trade-off. To further improve this, we alter the bagged-boosting scheme by introducing collaboration between the multi-model’s constituent learners at various levels. This novel stability-guided classification scheme is delivered in two flavours: during or after the boosting process. Applied among a crowd of Gentle Boost ensembles, the ability of the two suggested algorithms to generalize is inspected by comparing them against Subbagging and Gentle Boost on various real-world datasets. In both cases, our models obtained a 40% generalization error decrease. But their true ability to capture details in data was revealed through their application for protein detection in texture analysis of gel electrophoresis images. They achieve improved performance of approximately 0.9773 AUROC when compared to the AUROC of 0.9574 obtained by an SVM based on recursive feature elimination.

show abstract

Section: Resultssupporting

confidence: 70%

Generating highly accurate prediction hypotheses through collaborative ensemble learning

Arsov

Pavlovski

Basnarkov

et al. 2017

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Gao and Zhou (2010) introduce the notion of approximation stability, and prove that the approximation stability is sufficient for generalization and necessary for learnability of asymptotical empirical risk minimization (AERM). Then, Gao and Zhou (2010) prove that AdaBoost has approximation stability and thus good generalization. Moreover, an exponential bound for AdaBoost is provided.…”

Section: Boostingmentioning

confidence: 99%

“…One such view is due to Friedman et al (2000) who propose that boosting is stagewise additive model fitting. Gao and Zhou (2010) introduce the notion of approximation stability, and prove that the approximation stability is sufficient for generalization and necessary for learnability of asymptotical empirical risk minimization (AERM). Then, Gao and Zhou (2010) prove that AdaBoost has approximation stability and thus good generalization.…”

Section: Boostingmentioning

confidence: 99%

Bagging and boosting variants for handling classifications problems: a survey

Kotsiantis

2013

The Knowledge Engineering Review

View full text Add to dashboard Cite

Bagging and boosting are two of the most well-known ensemble learning methods due to their theoretical performance guarantees and strong experimental results. Since bagging and boosting are an effective and open framework, several researchers have proposed their variants, some of which have turned out to have lower classification error than the original versions. This paper tried to summarize these variants and categorize them into groups. We hope that the references cited cover the major theoretical issues, and provide access to the main branches of the literature dealing with such methods, guiding the researcher in interesting research directions.

show abstract