Toward efficient agnostic learning

Kearns, Michael; Schapire, Robert E.; Sellie, Linda

doi:10.1145/130385.130424

Cited by 226 publications

(191 citation statements)

References 19 publications

(14 reference statements)

Supporting

Mentioning

191

Contrasting

Order By: Relevance

“…2. This is in contrast to the case of probabilistic models of learning, where efficient algorithms with good learning performance have been discovered for this function class (Kearns, Schapire & Sellie, 1992).…”

Section: Acknowledgmentmentioning

confidence: 88%

Untitled

Auer¹,

Long²,

Maass³

et al. 1995

Machine Learning

View full text Add to dashboard Cite

Abstract. The majority of results in computational learning theory are concerned with concept learning, i.e. with the special case of function learning for classes of functions with range {0, 1}. Much less is known about the theory of learning functions with a larger range such as N or R. In particular relatively few results exist about the general structure of common models for function learning, and there are only very few nontrivial function classes for which positive learning results have been exhibited in any of these models.We introduce in this paper the notion of a binary branching adversary tree for function learning, which allows us to give a somewhat surprising equivalent characterization of the optimal learning cost for learning a class of real-valued functions (in terms of a max-min definition which does not involve any "learning" model).Another general structural result of this paper relates the cost for learning a union of function classes to the learning costs for the individual function classes.Furthermore, we exhibit an efficient learning algorithm for learning convex piecewise linear functions from R d into R. Previously, the class of linear functions from R d into R was the only class of functions with multidimensional domain that was known to be learnable within the rigorous framework of a formal model for online learning.Finally we give a sufficient condition for an arbitrary class f of functions from R into R that allows us to learn the class of all functions that can be written as the pointwise maximum of k functions from f. This allows us to exhibit a number of further nontrivial classes of functions from R into R for which there exist efficient learning algorithms.

show abstract

Section: Acknowledgmentmentioning

confidence: 88%

Untitled

Auer¹,

Long²,

Maass³

et al. 1995

Machine Learning

View full text Add to dashboard Cite

show abstract

“…This is a different task than agnostic learning [11], which requires the learner to find a concept that best approximates the probabilistic (noisy) concept observed. An extra difficulty of our task arises from the fact that the noise process may generate two similar probabilistic concepts from two dissimilar concepts.…”

Section: Class Noise Modelsmentioning

confidence: 99%

PAC-Learning with General Class Noise Models

Jabbari

Holte

Zilles

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We introduce a framework for class noise, in which most of the known class noise models for the PAC setting can be formulated. Within this framework, we study properties of noise models that enable learning of concept classes of finite VC-dimension with the Empirical Risk Minimization (ERM) strategy. We introduce simple noise models for which classical ERM is not successful. Aiming at a more generalpurpose algorithm for learning under noise, we generalize ERM to a more powerful strategy. Finally, we study general characteristics of noise models that enable learning of concept classes of finite VC-dimension with this new strategy.

show abstract

“…A well established approach in such cases (Vovk, 1990;Littlestone & Warmuth, 1994;Littlestone, 1989;Feder, Merhav, & Gutman, 1992;Merhav & Feder, 1993;Cesa-Bianchi et al, 1997;Cesa-Bianchi et al, 1996) is to assume nothing about the (x t , y t ) pairs, and instead, for a given F, to give bounds on the number of mistakes made by a given learning algorithm in terms of the minimum over f ∈ F of the number η of trials t for which f (x t ) = y t . Learning models like this are often referred to as agnostic learning models 4 (Kearns, Schapire, & Sellie, 1994).…”

Section: Agnostic Learningmentioning

confidence: 99%

Untitled

Auer

Long

1999

Machine Learning

View full text Add to dashboard Cite

Abstract.We solve an open problem of Maass and Turán, showing that the optimal mistake-bound when learning a given concept class without membership queries is within a constant factor of the optimal number of mistakes plus membership queries required by an algorithm that can ask membership queries. Previously known results imply that the constant factor in our bound is best possible.We then show that, in a natural generalization of the mistake-bound model, the usefulness to the learner of arbitrary "yes-no" questions between trials is very limited. We show that several natural structural questions about relatives of the mistake-bound model can be answered through the application of this general result. Most of these results can be interpreted as saying that learning in apparently less powerful (and more realistic) models is not much more difficult than learning in more powerful models.

show abstract

Toward efficient agnostic learning

Cited by 226 publications

References 19 publications

Untitled

Untitled

PAC-Learning with General Class Noise Models

Untitled

Contact Info

Product

Resources

About