Learnability with respect to fixed distributions

Balsters, Herman; Fokkinga, Maarten M.

doi:10.1016/0304-3975(91)90026-x

Cited by 104 publications

(91 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pair (C, H) is said to be passively learnable with respect to 1P if there exists a function f E FCH such that for every E, 6 > 0 there is a 0 < m(e, 6) < oo such that for every probability measure P E P7 and every c E C, if 7 E X m is chosen at random according to p m then the probability that errorf,c,p(T) < e is greater than 1 -6. If P is the set of all probability distributions over some fixed a-algebra of X (which we will denote by p*), then the above definition reduces to the version from Blumer et al [6] of Valiant's [20] original definition (without restrictions on computability) for learnability for all distributions. If P consists of a single distribution then the above definition reduces to that used by Benedek and Itai [5]. As often done in the literature, we will be considering the case H = C throughout, so that we will simply speak of learnability of C rather than learnability of (C, H).…”

Section: Definitions Of Passive and Active Learnabilitymentioning

confidence: 99%

“…That is, P consists of a single distribution P that is known to the learner. Benedek and Itai [5] obtained conditions for passive learnability in this case in terms of a quantity known as metric entropy.…”

Section: Active Learning For a Fixed Distributionmentioning

confidence: 99%

“…Benedek and Itai [5] proved that a concept class C is passively learnable for a fixed distribution P iff C has finite metric entropy with respect to P, and they provided upper and lower bounds on the number of samples required. Specifically, they showed that any passive learning algorithm requires at least log 2 (1 -6)N(2e, C, P) samples and that (32/e) ln(N(e/2)/6) samples is sufficient.…”

Section: Definition 3 (Metric Entropy) Let (Y P) Be a Metric Space mentioning

confidence: 99%

“…For example, learnability with respect to a class of distributions (as opposed to the original distribution-free framework) has been studied [5,13,14,15]. Notably, Benedek and Itai [5] first studied learnability with respect to a fixed and known probability distribution, and characterized learnability in this case in terms of the metric entropy of the concept class. Others have considered particular instances of learnability with respect to a fixed distribution.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Active Learning Using Arbitrary Binary Valued Queries

KulKarni¹,

Mitter²,

Tsitsiklis³

1990

View full text Add to dashboard Cite

The original and most widely studied PAC model for learning assumes a passive learner in the sense that the learner plays no role in obtaining information about the unknown concept. That is, the samples are simply drawn independently from some probability distribution. Some work has been done on studying more powerful oracles and how they affect learnability. To find bounds on the improvement that can be expected from using oracles, we consider active learning in the sense that the learner has complete choice in the information received. Specifically, we allow the learner to ask arbitrary yes/no questions. We consider both active learning under a fixed distribution and distribution-free active learning. In the case of active learning, the underlying probability distribution is used only to measure distance between concepts. For learnability with respect to a fixed distribution, active learning does not enlarge the set of learnable concept classes, but can improve the sample complexity. For distribution-free learning, it is shown that a concept class is actively learnable iff it is finite, so that active learning is in fact less powerful than the usual passive learning model. We also consider a form of distribution-free learning in which the learner knows the distribution being used, so that 'distribution-free' refers only to the requirement that a bound on the number of queries can be obtained uniformly over all distributions. Even with the side information of the distribution being used, a concept class is actively learnable iff it has finite VC dimension, so that active learning with the side information still does not enlarge the set of learnable concept classes.*''IThis work was supported by the U.S. Army Research Office under Contract DAAL03-86-K1-0171 , by the Department. of the Na.vy under Air Force Contract F119628-9U-C-0002, and by the National Science Foundation under con t ract ECS-8552419. Report Documentation PageForm Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

show abstract

Section: Definitions Of Passive and Active Learnabilitymentioning

confidence: 99%

Section: Active Learning For a Fixed Distributionmentioning

confidence: 99%

Section: Definition 3 (Metric Entropy) Let (Y P) Be a Metric Space mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active Learning Using Arbitrary Binary Valued Queries

KulKarni¹,

Mitter²,

Tsitsiklis³

1990

View full text Add to dashboard Cite

show abstract

“…Some variations/extensions of the original model that have been studied and are relevant to the present work include learning with respect to a fixed distribution [3,6], and learning functions as opposed to sets (i.e., binary valued functions) [6]. As the name suggests, learning with respect to a fixed distribution refers to the case in which the distribution with which the samples are being drawn is fixed and known to the learner.…”

Section: Pac Learning With Generalized Samplesmentioning

confidence: 99%

PAC Learning with Generalized Samples and an Application to Stochastic Geometry

Kulkarni¹,

Mitter²,

Tsitsiklis³

et al. 1991

View full text Add to dashboard Cite

In this paper, we introduce an extension of the standard PAC model which allows the use of generalized samples. We view a generalized sample as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. We consider a specific application of the generalized model to a problem of curve reconstruction, and discuss some connections with a result from stochastic geometry.

show abstract

FIR Volterra kernel neural models and PAC learning

Najarian

2002

Complexity

View full text Add to dashboard Cite

The probably approximately correct (PAC) learning theory creates a framework to assess the learning properties of static models for which the data are assumed to be independently and identically distributed (i.i.d.). The present article first extends the idea of PAC learning to cover the learning of modeling tasks with m-dependent sequences of data. The data are assumed to be marginally distributed according to a fixed arbitrary probability.The resulting framework is then applied to evaluate learning of Volterra Kernel FIR models.

show abstract

Learnability with respect to fixed distributions

Cited by 104 publications

References 8 publications

Active Learning Using Arbitrary Binary Valued Queries

Active Learning Using Arbitrary Binary Valued Queries

PAC Learning with Generalized Samples and an Application to Stochastic Geometry

FIR Volterra kernel neural models and PAC learning

Contact Info

Product

Resources

About