Bolstered error estimation

Braga-Neto, Ulisses; Dougherty, Edward R.

doi:10.1016/j.patcog.2003.08.017

Cited by 112 publications

(81 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These conclusions are also supported by Figures 3 and 4. We observed that the performance of error estimators other than out-of-bag (which can only be applied to ensemble rules) were consistent with their performance with the corresponding single classifier, as reported in other studies [18,27].…”

Section: Synthetic Datasupporting

confidence: 89%

“…Bolstered estimation was proposed in [27]. It has shown promising performance for small sample sizes in terms of root mean square error.…”

Section: Bolstered Error Estimatorsmentioning

confidence: 99%

“…Semibolstering is suggested for correcting this, by conducting no bolstering at misclassified points. We refer the reader to [27] for the full details (in this paper, we employ the bolstered and semibolstered resubstitution estimators of [27]). …”

Section: Bolstered Error Estimatorsmentioning

confidence: 99%

See 2 more Smart Citations

Small-Sample Error Estimation for Bagged Classification Rules

Braga-Neto

2010

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Application of ensemble classification rules in genomics and proteomics has become increasingly common. However, the problem of error estimation for these classification rules, particularly for bagging under the small-sample settings prevalent in genomics and proteomics, is not well understood. Breiman proposed the "out-of-bag" method for estimating statistics of bagged classifiers, which was subsequently applied by other authors to estimate the classification error. In this paper, we give an explicit definition of the out-of-bag estimator that is intended to remove estimator bias, by formulating carefully how the error count is normalized. We also report the results of an extensive simulation study of bagging of common classification rules, including LDA, 3NN, and CART, applied on both synthetic and real patient data, corresponding to the use of common error estimators such as resubstitution, leave-one-out, cross-validation, basic bootstrap, bootstrap 632, bootstrap 632 plus, bolstering, semi-bolstering, in addition to the out-of-bag estimator. The results from the numerical experiments indicated that the performance of the out-of-bag estimator is very similar to that of leave-one-out; in particular, the out-of-bag estimator is slightly pessimistically biased. The performance of the other estimators is consistent with their performance with the corresponding single classifiers, as reported in other studies.

show abstract

Section: Synthetic Datasupporting

confidence: 89%

“…Bolstered estimation was proposed in [27]. It has shown promising performance for small sample sizes in terms of root mean square error.…”

Section: Bolstered Error Estimatorsmentioning

confidence: 99%

See 1 more Smart Citation

Small-Sample Error Estimation for Bagged Classification Rules

Braga-Neto

2010

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

show abstract

“…The gene-label correlation reflects the predictive power, or relevancy, of a gene and could be used to identify biologically related genes of certain biological phenomenon of interest. However, the correlation criterion Eqn (1) evaluates genes on an individual basis, without considering correlations between genes. Severe redundancy might exist if it is used to select gene subsets.…”

Section: Relevancy and Redundancy Measuresmentioning

confidence: 99%

“…The various gene selection algorithms can be classified into two categories, namely individual gene selection (see for example [8,4,7,15]) and gene subset selection (see for example [14,11,6,10,12,21,20,1]). The two types of gene selection algorithms often serve different purposes.…”

Section: Introductionmentioning

confidence: 99%

Correlation-Based Relevancy and Redundancy Measures for Efficient Gene Selection

Mao

Tang

Pattern Recognition in Bioinformatics

View full text Add to dashboard Cite

Abstract. The gene-label correlation provides an effective measure of the relevancy of a gene. However, this measure evaluates genes on an individual basis, and the gene sets thus obtained may exhibit severe redundancy. In this study, we propose a new correlation heuristic for set-based gene selection, with the goal of alleviating the redundancy problem. The new correlation heuristic consists of two components that account for gene relevancy and redundancy respectively. The relevancy of a gene is evaluated in terms of its correlation with class label on an individual basis, while the redundancy of a gene with respect to a given gene subset is measured by its correlation with a new dimension built upon the gene subset. The new correlation heuristic retains the simplicity of individual gene evaluation and the redundancy handling capacity of set-based gene evaluation. Two different ways of using the relevancy and redundancy measures are presented in this study. One way is the maximization of the ratio of relevancy measure to redundancy measure, and another way is the maximization of the relevancy measure subtracting redundancy measure. Experimental studies on six gene expression problems show that both criteria produce excellent results.

show abstract

Microarray Data Analysis

Moffitt

Wang

2006

Wiley Encyclopedia of Biomedical Engineering

View full text Add to dashboard Cite

Microarray analysis is an emerging field, simultaneously harnessing advances in semiconductor manufacturing, biochemistry, medicine, computation, and algorithms research. Microarrays now provide a platform for an unprecedented genome‐wide view of a biological sample. Microarray analysis makes use of the vast amounts of data that the microarray platform provides. It is through the intelligent combination of mathematical algorithms and clinical validation that microarray analysis provides a real opportunity to realize the goal of targeted personalized medicine. One day, the information from a single microarray might be able to tell a doctor if a patient has cancer, what type of cancer it is, what the prognosis is, and what drug to use to best fight the cancer. The foundation of this story is being built in laboratories across the world today and it starts with sound microarray analysis. Microarray analysis is a multistep process that converts raw microarray data into biomarkers for clinical use. First, noise must be removed from raw data using preprocessing methods, such as normalization and artifact removal. Clean data can then be used to select important features or to build predictive rules called classifiers. The results of feature selection and classification are lists of biomarkers that are appropriate for classifying the data into groups such as benign or malignant. These biomarkers must then be validated clinically or through knowledge‐based approaches. The results of validation can then be used as feedback in order to select better features or build better classifiers.

show abstract

Bolstered error estimation

Cited by 112 publications

References 13 publications

Small-Sample Error Estimation for Bagged Classification Rules

Small-Sample Error Estimation for Bagged Classification Rules

Correlation-Based Relevancy and Redundancy Measures for Efficient Gene Selection

Microarray Data Analysis

Contact Info

Product

Resources

About