Toward efficient agnostic learning

Kearns, Michael; Schapire, Robert E.; Sellie, Linda

doi:10.1007/bf00993468

Cited by 142 publications

(66 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When an algorithm is run on a sequence of feature vectors and rewards satisfying these assumptions, we call it an η-admissible run of the algorithm. Following [23], we refer to this as the agnostic case.…”

Section: The Agnostic Casementioning

confidence: 99%

Reinforcement Learning with Immediate Rewards and Linear Hypotheses

Abe¹,

Biermann

Long

2003

Algorithmica

View full text Add to dashboard Cite

We consider the design and analysis of algorithms that learn from the consequences of their actions with the goal of maximizing their cumulative reward, when • the consequence of a given action is felt immediately, and • a linear function, which is unknown a priori, (approximately) relates a feature vector for each action/state pair to the (expected) associated reward.We focus on two cases, one in which a continuous-valued reward is (approximately) given by applying the unknown linear function, and another in which the probability of receiving the larger of binary-valued rewards is obtained. For these cases we provide bounds on the per-trial regret for our algorithms that go to zero as the number of trials approaches infinity. We also provide lower bounds that show that the rate of convergence is nearly optimal.

show abstract

Section: The Agnostic Casementioning

confidence: 99%

Reinforcement Learning with Immediate Rewards and Linear Hypotheses

Abe¹,

Biermann

Long

2003

Algorithmica

View full text Add to dashboard Cite

show abstract

“…The agnostic setting It is well-known that a hypothesis class H is PAC-learnable in the (so-called) agnostic setting (with no a-priori assumptions about the data) iff the corresponding Minimum Disagreement Problem for H can be solved in polynomial time [15]. The completely analogous remark holds for "Minimum Onesided Disagreement" and "Agnostic PAC-learning with One-sided Empirical Error".…”

Section: Some Closing Remarksmentioning

confidence: 74%

PAC-learning in the presence of one-sided classification noise

Simon

2012

Ann Math Artif Intell

View full text Add to dashboard Cite

We derive an upper and a lower bound on the sample size needed for PAClearning a concept class in the presence of one-sided classification noise. The upper bound is achieved by the strategy "Minimum One-sided Disagreement". It matches the lower bound (which holds for any learning strategy) up to a logarithmic factor. Although "Minimum One-sided Disagreement" often leads to NP-hard combinatorial problems, we show that it can be implemented quite efficiently for some simple concept classes like, for example, unions of intervals, axis-parallel rectangles, and TREE(2, n, 2, k) which is a broad subclass of 2-level decision trees. For the first class, there is an easy algorithm with time bound O(m log m). For the second-one (resp. the third-one), we design an algorithm that applies the well-known UNION-FIND data structure and has an almost quadratic time bound (resp. time bound O(n 2 m log m)).

show abstract

“…An O(n 6 k)-time algorithm for finding one polygon in the class of convex s-gons that minimizes the classification error on a sample of labeled points is given [31]. This result implies that convex polygons are learnable in the PAC model with random classification noise [32] and in the "agnostic" PAC model [33].…”

Section: Learning In Constant Dimension For Constant Dimension Baummentioning

confidence: 98%

PAC Learning Intersections of Halfspaces with Membership Queries

Kwek¹,

Pitt²

1998

Algorithmica

View full text Add to dashboard Cite

A randomized learning algorithm POLLY is presented that efficiently learns intersections of s halfspaces in n dimensions, in time polynomial in both s and n. The learning protocol is the PAC (probably approximately correct) model of Valiant, augmented with membership queries. In particular, POLLY receives a set S of m = poly(n, s, 1/ε, 1/δ) randomly generated points from an arbitrary distribution over the unit hypercube, and is told exactly which points are contained in, and which points are not contained in, the convex polyhedron P defined by the halfspaces. POLLY may also obtain the same information about points of its own choosing. It is shown that after poly(n, s, 1/ε, 1/δ, log(1/d)) time, the probability that POLLY fails to output a collection of s halfspaces with classification error at most ε, is at most δ. Here, d is the minimum distance between the boundary of the target and those examples in S that are not lying on the boundary. The parameter log(1/d) can be bounded by the number of bits needed to encode the coefficients of the bounding hyperplanes and the coordinates of the sampled examples S. Moreover, POLLY can be extended to learn unions of k disjoint polyhedra with each polyhedron having at most s facets, in time poly(n, k, s, 1/ε, 1/δ, log(1/d), 1/γ ) where γ is the minimum distance between any two distinct polyhedra.

show abstract

Toward efficient agnostic learning

Cited by 142 publications

References 23 publications

Reinforcement Learning with Immediate Rewards and Linear Hypotheses

Reinforcement Learning with Immediate Rewards and Linear Hypotheses

PAC-learning in the presence of one-sided classification noise

PAC Learning Intersections of Halfspaces with Membership Queries

Contact Info

Product

Resources

About