Combinatorial variability of vapnik-chervonenkis classes with applications to sample compression schemes

Ben-David, Shai; Litman, Ami

doi:10.1016/s0166-218x(98)00000-6

Cited by 31 publications

(36 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It also has the property to reduce to the Occam's razor bound when the compression set z i vanishes. The idea of using a message string as an additional source of information was also used by Littlestone and Warmuth (1986) and Ben-David and Litman (1998) to obtain a sample-compression bound looser than the bound presented here. Moreover, in contrast with these bounds, Theorem 1 applies to any compression set-dependent distribution of messages P M(z i ) satisfying:…”

Section: A Data-compression Risk Boundmentioning

confidence: 99%

Margin-Sparsity Trade-Off for the Set Covering Machine

Laviolette

Marchand

Shah

2005

Machine Learning: ECML 2005

View full text Add to dashboard Cite

show abstract

Section: A Data-compression Risk Boundmentioning

confidence: 99%

Margin-Sparsity Trade-Off for the Set Covering Machine

Laviolette

Marchand

Shah

2005

Machine Learning: ECML 2005

View full text Add to dashboard Cite

show abstract

“…We call two concept classes C, C equivalent if their matrices are equal up to permutation of rows or columns, and up to flipping all bits of a subset of the rows. 4 The following result characterizes the classes of recursive teaching dimension 1:…”

Section: Recursive Teaching and Query Learningmentioning

confidence: 99%

Recursive Teaching Dimension, Learning Complexity, and Maximum Classes

Doliwa

Simon

Zilles

2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter in this context. Comparing the RTD to self-directed learning, we establish new lower bounds on the query complexity for a variety of query learning models and thus connect teaching to query learning. For many general cases, the RTD is upper-bounded by the VC-dimension, e.g., classes of VC-dimension 1, (nested differences of) intersection-closed classes, "standard" boolean function classes, and finite maximum classes. The RTD thus is the first model to connect teaching to the VC-dimension. The combinatorial structure defined by the RTD has a remarkable resemblance to the structure exploited by sample compression schemes and hence connects teaching to sample compression. Sequences of teaching sets defining the RTD coincide with unlabeled compression schemes both (i) resulting from Rubinstein and Rubinstein's corner-peeling and (ii) resulting from Kuzmin and Warmuth's Tail Matching algorithm.

show abstract

“…Lemma 2 (Ben-David and Litman [2]). Dudley classes of dimension k are embeddable in maximum classes of VC-dimension k.…”

Section: Some Popular Examples Of Dudley Classes Includementioning

confidence: 99%

“…The precise relationship between these two combinatorial parameters is hence of great interest to the learning theory community, as witnessed by a continuing series of publications on the topic, see, e.g., [2,4,5,8,11,12,14]. Floyd and Warmuth [8] conjectured that any concept class C of VC-dimension d It is even open whether or not the sample compression number is linear in the VC-dimension.…”

Section: Introductionmentioning

confidence: 99%

Order Compression Schemes

Darnstädt

Doliwa

Simon

et al. 2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Sample compression schemes are schemes for "encoding" a set of examples in a small subset of examples. The long-standing open sample compression conjecture states that, for any concept class C of VC-dimension d, there is a sample compression scheme in which samples for concepts in C are compressed to samples of size at most d. We show that every order over C induces a special type of sample compression scheme for C, which we call order compression scheme. It turns out that order compression schemes can compress to samples of size at most d if C is maximum, intersection-closed, a Dudley class, or of VCdimension 1-and thus in most cases for which the sample compression conjecture is known to be true. Since order compression schemes are much simpler than sample compression schemes in general, their study seems to be a promising step towards resolving the sample compression conjecture. We reveal a number of fundamental properties of order compression schemes, which are helpful in such a study. In particular, order compression schemes exhibit interesting graph-theoretic properties as well as connections to the theory of learning from teachers.

show abstract

Combinatorial variability of vapnik-chervonenkis classes with applications to sample compression schemes

Cited by 31 publications

References 7 publications

Margin-Sparsity Trade-Off for the Set Covering Machine

Margin-Sparsity Trade-Off for the Set Covering Machine

Recursive Teaching Dimension, Learning Complexity, and Maximum Classes

Order Compression Schemes

Contact Info

Product

Resources

About