A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

Freund, Yoav; Schapire, Robert E.

doi:10.1006/jcss.1997.1504

Cited by 14,586 publications

(8,979 citation statements)

References 16 publications

Supporting

Mentioning

8,464

Contrasting

Unclassified

101

Order By: Relevance

“…The process of building the model uses a process known as boosting (AdaBoost algorithm [28]) to enhance the predictive capability. Boosting combined together several weighted individual predictors into one model, with each individual predictor composed of the LNTCP metric and a decision tree.…”

Section: Introductionmentioning

confidence: 99%

Predicting Lung Radiotherapy-Induced Pneumonitis Using a Model Combining Parametric Lyman Probit With Nonparametric Decision Trees

Das¹,

Zhou²,

Zhang³

et al. 2007

International Journal of Radiation Oncology*Biology*Physics

View full text Add to dashboard Cite

Purpose: To develop and test a model to predict for lung radiation-induced Grade 2+ pneumonitis. Methods and Materials:The model was built from a database of 234 lung cancer patients treated with radiotherapy (RT), of whom 43 were diagnosed with pneumonitis. The model augmented the predictive capability of the parametric dose-based Lyman normal tissue complication probability (LNTCP) metric by combining it with weighted nonparametric decision trees that use dose and nondose inputs. The decision trees were sequentially added to the model using a "boosting" process that enhances the accuracy of prediction. The model's predictive capability was estimated by 10-fold cross-validation. To facilitate dissemination, the cross-validation result was used to extract a simplified approximation to the complicated model architecture created by boosting. Application of the simplified model is demonstrated in two example cases. Results:The area under the model receiver operating characteristics curve for cross-validation was 0.72, a significant improvement over the LNTCP area of 0.63 (p = 0.005). The simplified model used the following variables to output a measure of injury: LNTCP, gender, histologic type, chemotherapy schedule, and treatment schedule. For a given patient RT plan, injury prediction was highest for the combination of pre-RT chemotherapy, once-daily treatment, female gender and lowest for the combination of no pre-RT chemotherapy and nonsquamous cell histologic type. Application of the simplified model to the example cases revealed that injury prediction for a given treatment plan can range from very low to very high, depending on the settings of the nondose variables.Conclusions: Radiation pneumonitis prediction was significantly enhanced by decision trees that added the influence of nondose factors to the LNTCP formulation.

show abstract

Section: Introductionmentioning

confidence: 99%

Predicting Lung Radiotherapy-Induced Pneumonitis Using a Model Combining Parametric Lyman Probit With Nonparametric Decision Trees

Das¹,

Zhou²,

Zhang³

et al. 2007

International Journal of Radiation Oncology*Biology*Physics

View full text Add to dashboard Cite

show abstract

“…It has been shown that provided enough data for training, weak classifiers (performing slightly better than random guessing) can be aggregated to form an arbitrarily good classifier satisfying any desired (classification or generalization) error tolerance. Schapire introduced the first polynomial time boosting algorithm in 1990, and in 1995, Freund and Schapire (1996 introduced the Adaboost algorithm for binary classification (see also Schapire, 1999), and, later, versions for multiclass classification (Allwein et al, 2000). Mason et al (1999) introduced a general framework for Boosting design based on convex risk minimization with arbitrary convex loss function.…”

mentioning

confidence: 99%

fMRI pattern classification using neuroanatomically constrained boosting

et al. 2006

View full text Add to dashboard Cite

Pattern classification in functional MRI (fMRI) is a novel methodology to automatically identify differences in distributed neural substrates resulting from cognitive tasks. Reliable pattern classification is challenging due to the high dimensionality of fMRI data, the small number of available data sets, interindividual differences, and dependence on the acquisition methodology. Thus, most previous fMRI classification methods were applied in individual subjects. In this study, we developed a novel approach to improve multiclass classification across groups of subjects, field strengths, and fMRI methods. Spatially normalized activation maps were segmented into functional areas using a neuroanatomical atlas and each map was classified separately using local classifiers. A single multiclass output was applied using a weighted aggregation of the classifier's outputs. An Adaboost technique was applied, modified to find the optimal aggregation of a set of spatially distributed classifiers. This Adaboost combined the regionspecific classifiers to achieve improved classification accuracy with respect to conventional techniques. Multiclass classification accuracy was assessed in an fMRI group study with interleaved motor, visual, auditory, and cognitive task design. Data were acquired across 18 subjects at different field strengths (1.5 T, 4 T), with different pulse sequence parameters (voxel size and readout bandwidth). Misclassification rates of the boosted classifier were between 3.5% and 10%, whereas for the single classifier, these were between 15% and 23%, suggesting that the boosted classifier provides a better generalization ability together with better robustness. The high computational speed of boosting classification makes it attractive for real-time fMRI to facilitate online interpretation of dynamically changing activation patterns.Keywords: Functional magnetic resonance imaging; Pattern classification; Support vector machines; Adaboost IntroductionBrain activation changes in response to even simple sensory input and motor tasks encompass a widely distributed network of functional brain areas. Information embedded in the spatial shape and extent of these activation patterns, and differences in voxel-tovoxel time course, are not easily quantified with conventional analysis tools, such as statistical parametric mapping (SPM) (Kiebel and Friston, 2004a,b). Pattern classification in functional MRI (fMRI) is a novel approach, which promises to characterize subtle differences in activation patterns between different tasks. However, automatic and reliable classification of patterns is challenging due to the high dimensionality of fMRI data, the small number of available data sets, interindividual differences in activation patterns, and dependence on the image acquisition methodology. Recent work by Cox and Savoy (2003) demonstrated that linear discriminant analysis and support vector machines (SVM) allow 10-way discrimination of visual activation patterns evoked by the visual presentation of various categories of ...

show abstract

“…24. Classification model using the aforementioned features was constructed with three different state-of-the-art ensemble and meta classifiers: Random forest (25), Bagging (26) and LogitBoost (27)(28)(29)(30). The performance of the model was evaluated based on the promoter prediction metrics suggested by (31): sensitivity (SN), positive predictive value (PPV), correlation coefficient (CC) and true-positive cost (TPC).…”

Section: Identification and Annotation Of Pol-ii Promoter Peaksmentioning

confidence: 99%