Active Class Selection

Lomasky, R.; Brodley, Carla E.; Aernecke, Matthew J.; Walt, David R.; Friedl, M. A.

doi:10.1007/978-3-540-74958-5_63

Cited by 39 publications

(36 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consider as a motivational example, the problem of building predictive models based on data collected through an "artificial nose" with the intent of "sniffing out" explosive or hazardous chemical compounds [38][39][40]. In this setting, the reactivity of a large number of chemicals is already known, representing label-conditioned pools of available instances.…”

Section: Class-conditional Example Acquisitionmentioning

confidence: 99%

“…Let us call the problem of proportioning class labels in a selection of n additional training instances, "active class selection" (ACS) [38][39][40]43]. This process is exemplified in Figure 6.14.…”

Section: Class-conditional Example Acquisitionmentioning

confidence: 99%

“…This family of techniques for performing ACS is based on the volatility in the predictions made about certain classes-those classes whose cross-validated predictions are subject to the most change between successive epochs of instance selection are likely to be based on an uncertain predictor and amenable to refinement by the incorporation of additional training data [38,40]. Analogous to the case of more traditional uncertainty-based data acquisition, several heuristics have been devised to capture the notion of variability.…”

Section: Uncertainty-based Approachesmentioning

confidence: 99%

“…We can think that as the predictor gains knowledge of the problem space, it tends to solidify in the face of data, exhibiting less change and greater certainty. For ACS, we might wonder if the learning curves will be equally steep regardless of the class of the training data [38][39][40]. With this in mind, we can select instances at epoch t from the classes in proportion to their improvements in accuracy at t − 1 and t − 2.…”

Section: Uncertainty-based Approachesmentioning

confidence: 99%

“…Again, using cross-validated predictions at successive epochs, it is possible to isolate members of each class, and observe changes in the predicted class for each instance. For example, when the predicted label of a given instance changes between successive epochs, we can deem the instance to have been redistricted [38][39][40]. Again considering the level of volatility in a model's predictions to be a measurement of uncertainty, we can sample classes at epoch t according to each classes' proportional measure of redistricting:…”

Section: Uncertainty-based Approachesmentioning

confidence: 99%

See 4 more Smart Citations

Class Imbalance and Active Learning

2013

Imbalanced Learning

View full text Add to dashboard Cite

Abstract:The performance of a predictive model is tightly coupled with the data used during training. While using more examples in the training will often result in a better informed, more accurate model; limits on computer memory and real-world costs associated with gathering labeled examples often constrain the amount of data that can be used for training. In settings where the number of training examples is limited, it often becomes meaningful to carefully see just which examples are Q1 selected. In active learning (AL), the model itself places a hands-on role in the selection of examples for labeling from a large pool of unlabeled examples. These examples are used for model training. Numerous studies have demonstrated, both empirically and theoretically, the benefits of AL: Given a fixed budget, a training system that interactively involves the current model in selecting the training examples can often result in a far greater accuracy than a system that simply selects random training examples. Imbalanced settings provide special opportunities and challenges for AL. For example, while AL can be used to build models that counteract the harmful effects of learning under class imbalance, extreme class imbalance can cause an AL strategy to "fail," preventing the selection scheme from choosing any useful examples for labeling. This chapter focuses on the interaction between AL and class imbalance, discussing (i) AL techniques designed specifically for dealing with imbalanced settings, (ii) strategies that leverage AL to overcome the deleterious effects of class imbalance, (iii) how extreme class imbalance can prevent AL systems from selecting useful examples, and alternatives to AL in these cases.

show abstract

Section: Class-conditional Example Acquisitionmentioning

confidence: 99%

Section: Class-conditional Example Acquisitionmentioning

confidence: 99%

Section: Uncertainty-based Approachesmentioning

confidence: 99%

Section: Uncertainty-based Approachesmentioning

confidence: 99%

Section: Uncertainty-based Approachesmentioning

confidence: 99%

See 3 more Smart Citations

Class Imbalance and Active Learning

2013

Imbalanced Learning

View full text Add to dashboard Cite

show abstract

Active Learning in Recommender Systems

Rubens

Kaplan

Sugiyama

2010

Recommender Systems Handbook

View full text Add to dashboard Cite

Active Selection of Classification Features

Kok

Brouwer

Mandl

et al. 2021

Advances in Intelligent Data Analysis XIX

View full text Add to dashboard Cite

Some data analysis applications comprise datasets, where explanatory variables are expensive or tedious to acquire, but auxiliary data are readily available and might help to construct an insightful training set. An example is neuroimaging research on mental disorders, specifically learning a diagnosis/prognosis model based on variables derived from expensive Magnetic Resonance Imaging (MRI) scans, which often requires large sample sizes. Auxiliary data, such as demographics, might help in selecting a smaller sample that comprises the individuals with the most informative MRI scans. In active learning literature, this problem has not yet been studied, despite promising results in related problem settings that concern the selection of instances or instance-feature pairs. Therefore, we formulate this complementary problem of Active Selection of Classification Features (ASCF): Given a primary task, which requires to learn a model f : x → y to explain/predict the relationship between an expensive-to-acquire set of variables x and a class label y. Then, the ASCF-task is to use a set of readily available selection variables z to select these instances, that will improve the primary task's performance most when acquiring their expensive features x and including them to the primary training set.We propose two utility-based approaches for this problem, and evaluate their performance on three public real-world benchmark datasets. In addition, we illustrate the use of these approaches to efficiently acquire MRI scans in the context of neuroimaging research on mental disorders, based on a simulated study design with real MRI data.

show abstract

Active Class Selection

Cited by 39 publications

References 16 publications

Class Imbalance and Active Learning

Class Imbalance and Active Learning

Active Learning in Recommender Systems

Active Selection of Classification Features

Contact Info

Product

Resources

About