Agnostic active learning

Balcan, Maria-Florina; Beygelzimer, Alina; Langford, John

doi:10.1016/j.jcss.2008.07.003

Cited by 218 publications

(255 citation statements)

References 25 publications

Supporting

Mentioning

249

Contrasting

Order By: Relevance

“…As was shown by [8], active learning in this setting is particularly challenging, and a long line of work, from theoretical to practical, has specifically addressed this setting [3], [9], [10], [4], [15], [2].…”

Section: Related Workmentioning

confidence: 99%

“…There is a great deal of literature on active learning [7], [9], [1], [13]. The problem is tractable when the oracle always outputs the ground truth labels [7], in which case, a generalized binary search-style approach results in an estimation error that decays exponentially with the number of labels.…”

Section: Introductionmentioning

confidence: 99%

“…[6] provides algorithms when the oracle outputs incorrect labels with higher probability closer to the decision boundary of the ground truth hypothesis. Finally, a long line of work [1], [9], [10], [2], [15] studies agnostic active learning, where a certain fraction of the labels can have arbitrary bias.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active learning from noisy and abstention feedback

Yan

Chaudhuri

Javidi

2015

2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

An active learner is given an instance space, a label space and a hypothesis class, where one of the hypotheses in the class assigns ground truth labels to instances. Additionally, the learner has access to a labeling oracle, which it can interactively query for the label of any example in the instance space. The goal of the learner is to find a good estimate of the hypothesis in the hypothesis class that generates the ground truth labels while making as few interactive queries to the oracle as possible.This work considers a more general setting where the labeling oracle can abstain from providing a label in addition to returning noisy labels. We provide a model for this setting where the abstention rate and the noise rate increase as we get closer to the decision boundary of the ground truth hypothesis. We provide an algorithm and an analysis of the number of queries it makes to the labeling oracle; finally we provide matching lower bounds to demonstrate that our algorithm has near-optimal estimation accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Active learning from noisy and abstention feedback

Yan

Chaudhuri

Javidi

2015

2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

show abstract

“…The labels for messages, which on the average appear on the wrong side of the boundary, are flipped and a final SVM model is trained using the modified data. We note that RHT expands on agnostic active (A2) learning [44] [45] that maintains both a current version space and a region of uncertainty. Two data sets, TREC 2007 and CEAS 2008 were used for comparing the performance of RHT-SVM to the performance of Reject On Negative Impact (RONI) [5] as well as the performance of an SVM trained on the tainted training data.…”

Section: Semantics and Stratificationmentioning

confidence: 99%

Cyberspace Security Using Adversarial Learning and Conformal Prediction

Wechsler¹

2015

IIM

View full text Add to dashboard Cite

This paper advances new directions for cyber security using adversarial learning and conformal prediction in order to enhance network and computing services defenses against adaptive, malicious, persistent, and tactical offensive threats. Conformal prediction is the principled and unified adaptive and learning framework used to design, develop, and deploy a multi-faceted self-managing defensive shield to detect, disrupt, and deny intrusive attacks, hostile and malicious behavior, and subterfuge. Conformal prediction leverages apparent relationships between immunity and intrusion detection using non-conformity measures characteristic of affinity, a typicality, and surprise, to recognize patterns and messages as friend or foe and to respond to them accordingly. The solutions proffered throughout are built around active learning, meta-reasoning, randomness, distributed semantics and stratification, and most important and above all around adaptive Oracles. The motivation for using conformal prediction and its immediate off-spring, those of semi-supervised learning and transduction, comes from them first and foremost supporting discriminative and non-parametric methods characteristic of principled demarcation using cohorts and sensitivity analysis to hedge on the prediction outcomes including negative selection, on one side, and providing credibility and confidence indices that assist meta-reasoning and information fusion.

show abstract

“…While margin-based active learning remains by far the most popular formalism despite a lack of strong performance guarantees, there have been several recent works examining active learning based upon the PAC learning model [74] for realizable concept classes [7,16,25,27,37] and the agnostic learning model [42] for broader concept classes [5,6,26,38]. However, while these results are important, it should also be noted that they make assumptions which render them generally less applicable to the complex applications where active learning in most useful, although this is certainly a direction for future study.…”

Section: Related Workmentioning

confidence: 99%

Margin-based active learning for structured predictions

Small

Roth

2010

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Margin-based active learning remains the most widely used active learning paradigm due to its simplicity and empirical successes. However, most works are limited to binary or multiclass prediction problems, thus restricting the applicability of these approaches to many complex prediction problems where active learning would be most useful. For example, machine learning techniques for natural language processing applications often require combining multiple interdependent prediction problemsgenerally referred to as learning in structured output spaces. In many such application domains, complexity is further managed by decomposing a complex prediction into a sequence of predictions where earlier predictions are used as input to later predictions-commonly referred to as a pipeline model. This work describes methods for extending existing margin-based active learning techniques to these two settings, thus increasing the scope of problems for which active learning can be applied. We empirically validate these proposed active learning techniques by reducing the annotated data requirements on multiple instances of synthetic data, a semantic role labeling task, and a named entity and relation extraction system.

show abstract

Agnostic active learning

Cited by 218 publications

References 25 publications

Active learning from noisy and abstention feedback

Active learning from noisy and abstention feedback

Cyberspace Security Using Adversarial Learning and Conformal Prediction

Margin-based active learning for structured predictions

Contact Info

Product

Resources

About