Informative sampling for large unbalanced data sets

Lu, Zhenyu; Rughani, Anand I.; Tranmer, Bruce I.; Bongard, Josh

doi:10.1145/1388969.1389020

Cited by 7 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Informative sampling [6] (henceforth referred to as IS) applies the EEA as an active learning method for classification. The algorithm works by iteratively evolving an ensemble of classifiers based on the current training set and scanning a portion of the whole data set to select the data point that causes maximal disagreement.…”

Section: Introduction and Methodsmentioning

confidence: 99%

“…The algorithm works by iteratively evolving an ensemble of classifiers based on the current training set and scanning a portion of the whole data set to select the data point that causes maximal disagreement. Although artificial neural networks [4] were chosen as the classifier type in [6], informative sampling is a general algorithm that could work with a wide range of classifiers. Many types of classifiers exist in the literature, such as decision trees, ANNs, and support vector machines (SVM).…”

Section: Introduction and Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting multiple classifier types with active learning

Bongard

2009

Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

Many approaches to active learning involve training one classifier by periodically choosing new data points about which the classifier has the least confidence, but designing a confidence measure without bias is nontrivial. An alternative approach is to train an ensemble of classifiers by periodically choosing data points that cause maximal disagreement among them. Many classifiers with different underlying structures could fit this framework, but some classifiers are more suitable for different data sets than others. The question then arises as to how to find the most suitable classifier for a given data set. In this work, an evolutionary algorithm is proposed to address this problem. The algorithm starts with a combination of artificial neural networks and decision trees, and iteratively adapts the ratio of the classifier types according to a replacement strategy. Experiments with synthetic and real data sets show that when the algorithm considers both fitness and classifier type for replacement, the population becomes saturated with accurate instantiations of the more suitable classifier type. This allows the algorithm to perform consistently well across data sets, without having to determine a priori a suitable classifier type.

show abstract

Section: Introduction and Methodsmentioning

confidence: 99%

Section: Introduction and Methodsmentioning

confidence: 99%

Exploiting multiple classifier types with active learning

Bongard

2009

Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Informative sampling [13] applied the EEA as an active learning method for classification. The algorithm works by iteratively optimizing an ensemble of classifiers based on the current training set and scanning a portion of the whole data set to select the data point that causes maximal disagreement among the label predictions of the current classifiers.…”

Section: Introductionmentioning

confidence: 99%

“…It has been shown to outperform random sampling [21] and balanced sampling [11] for a large unbalanced data set called the National Trauma Data Bank (NTDB) 1 . Although artificial neural networks (ANN) [10] were chosen as the classifier type in [13], informative sampling is a general algorithm that could work with any classifier. Many types of classi-fiers exist in the literature, such as decision trees (DT) [16], ANNs [10], and SVMs.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Informative Sampling for Active Learning

Lu¹,

Wu²,

Bongard³

2010

Proceedings of the 2010 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

Many approaches to active learning involve periodically training one classifier and choosing data points with the lowest confidence, but designing a confidence measure is nontrivial. An alternative approach is to periodically choose data instances that maximize disagreement among the label predictions across an ensemble of classifiers. Many classifiers with different underlying structures could fit this framework, but some ensembles are more suitable for some data sets than others. The question then arises as to how to find the most suitable ensemble for a given data set. In this work we introduce a method that begins with a heterogeneous ensemble composed of multiple instances of different classifier types, which we call adaptive informative sampling. The algorithm periodically adds data points to the training set, adapts the ratio of classifier types in the heterogeneous ensemble in favor of the better classifier type, and optimizes the classifiers in the ensemble using stochastic methods. Experimental results show that the proposed method performs consistently better than homogeneous ensembles. Comparison with random sampling and uncertainty sampling shows that the algorithm effectively draws informative data points for training.

show abstract

“…In a different approach, the estimation-exploration algorithm [19] [20][21](EEA) uses stochastic optimization algorithms to optimize models and find disagreement-causing data points. Informative sampling [22] adapted the EEA for data mining, but its performance is constrained by the limited power of stochastic optimization. For some classifiers, a satisfactory stochastic optimization algorithm may not exist, and existing classifiers such as C4.5 and Naive Bayes [23] can not currently be used in this framework.…”

mentioning

confidence: 99%

Active Learning with Adaptive Heterogeneous Ensembles

Bongard

2009

2009 Ninth IEEE International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

Abstract-One common approach to active learning is to iteratively train a single classifier by choosing data points based on its uncertainty, but it is nontrivial to design uncertainty measures unbiased by the choice of classifier. Query by committee [1] suggests that given an ensemble of diverse but accurate classifiers, the most informative data points are those that cause maximal disagreement among the predictions of the ensemble members. However the method for finding ensembles appropriate to a given data set remains an open question. In this paper, the random subspace method is combined with active learning to create multiple instances of different classifier types, and an algorithm is introduced that adapts the ratio of different classifier types in the ensemble towards better overall accuracy. Here we show that the proposed algorithm outperforms C4.5 with uncertainty sampling, Naive Bayes with uncertainty sampling, bagging, boosting and the random subspace method with random sampling. To the best of our knowledge, our work is the first to adapt the ratio of classifiers in a heterogeneous ensemble for active learning.

show abstract

Informative sampling for large unbalanced data sets

Cited by 7 publications

References 29 publications

Exploiting multiple classifier types with active learning

Exploiting multiple classifier types with active learning

Adaptive Informative Sampling for Active Learning

Active Learning with Adaptive Heterogeneous Ensembles

Contact Info

Product

Resources

About