L-norm Sauer–Shelah lemma for margin multi-category classifiers

Guermeur, Yann

doi:10.1016/j.jcss.2017.06.003

Cited by 14 publications

(50 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Realizing it at the level of a Rademacher complexity, a linear dependency on C was obtained in [8]. In this paper, as in [9], we showed that postponing it to the level of metric entropy, this dependency can be improved to a sublinear one. The case that remains to be studied is a decomposition at the level of a combinatorial dimension, more precisely, at that of the fat-shattering dimension.…”

Section: Discussionsupporting

confidence: 54%

“…Finally, a combinatorial bound gives an estimate on the metric entropy in terms of the combinatorial dimension. The metric entropy bound used in [9] is the L 2 -norm one of [12], which in this paper we generalized to L p -norms with integer p > 2. This generalization resulted in an improved dependency on the number C of categories compared to [9] without worsening the dependency on the sample size m nor the one on the margin parameter γ.…”

Section: Discussionmentioning

confidence: 99%

“…The major step that remains to perform to arrive at the claimed bound is to upper bound the right-hand side of (9). To this end, we appeal to Proposition 3.…”

Section: A Proof Of Theoremmentioning

confidence: 99%

“…Two families of margin loss functions can be distinguished: indicator margin loss functions and those that satisfy the Lipschitz condition. Deriving guaranteed risks with the optimal dependency on the parameters of interest is relatively straightforward in the first case [9]. The family of Lipschitz continuous loss functions, on the other hand, offers a richer setting to this task.…”

Section: Introductionmentioning

confidence: 99%

“…Performing a decomposition for Rademacher complexity, a linear dependency on C was obtained in [8]. This dependency has been improved to a sublinear one in [9] by postponing the decomposition to the level of metric entropy.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Rademacher complexity and generalization performance of multi-category margin classifiers

2019

Self Cite

View full text Add to dashboard Cite

One of the main open problems in the theory of multi-category margin classification is the form of the optimal dependency of a guaranteed risk on the number C of categories, the sample size m and the margin parameter γ. From a practical point of view, the theoretical analysis of generalization performance contributes to the development of new learning algorithms. In this paper, we focus only on the theoretical aspect of the question posed. More precisely, under minimal learnability assumptions, we derive a new risk bound for multi-category margin classifiers. We improve the dependency on C over the state of the art when the margin loss function considered satisfies the Lipschitz condition. We start with the basic supremum inequality that involves a Rademacher complexity as a capacity measure. This capacity measure is then linked to the metric entropy through the chaining method. In this context, our improvement is based on the introduction of a new combinatorial metric entropy bound.

show abstract

Section: Discussionsupporting

confidence: 54%

Section: Discussionmentioning

confidence: 99%

“…The major step that remains to perform to arrive at the claimed bound is to upper bound the right-hand side of (9). To this end, we appeal to Proposition 3.…”

Section: A Proof Of Theoremmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Rademacher complexity and generalization performance of multi-category margin classifiers

2019

Self Cite

View full text Add to dashboard Cite

show abstract

Rademacher complexity of margin multi-category classifiers

Guermeur

2018

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

One of the main open problems of the theory of margin multicategory pattern classification is the characterization of the optimal dependence of the confidence interval of a guaranteed risk on the three basic parameters which are the sample size m, the number C of categories and the scale parameter γ. This is especially the case when working under minimal learnability hypotheses. The starting point is a basic supremum inequality whose capacity measure depends on the choice of the margin loss function. Then, transitions are made, from capacity measure to capacity measure. At some level, a structural result performs the transition from the multi-class case to the bi-class one. In this article, we highlight the advantages and drawbacks inherent to the three major options for this decomposition: using Rademacher complexities, covering numbers or scale-sensitive combinatorial dimensions.

show abstract

Structure discovery in PAC-learning by random projections

Kabán,

Reeve

2024

Mach Learn

View full text Add to dashboard Cite

High dimensional learning is data-hungry in general; however, many natural data sources and real-world learning problems posses some hidden low-complexity structure that permit effective learning from relatively small sample sizes. We are interested in the general question of how to discover and exploit such hidden benign traits when problem-specific prior knowledge is insufficient. In this work, we address this question through random projection’s ability to expose structure. We study both compressive learning and high dimensional learning from this angle by introducing the notions of compressive distortion and compressive complexity. We give user-friendly PAC bounds in the agnostic setting that are formulated in terms of these quantities, and we show that our bounds can be tight when these quantities are small. We then instantiate these quantities in several examples of particular learning problems, demonstrating their ability to discover interpretable structural characteristics that make high dimensional instances of these problems solvable to good approximation in a random linear subspace. In the examples considered, these turn out to resemble some familiar benign traits such as the margin, the margin distribution, the intrinsic dimension, the spectral decay of the data covariance, or the norms of parameters—while our general notions of compressive distortion and compressive complexity serve to unify these, and may be used to discover benign structural traits for other PAC-learnable problems.

show abstract

L-norm Sauer–Shelah lemma for margin multi-category classifiers

Cited by 14 publications

References 30 publications

Rademacher complexity and generalization performance of multi-category margin classifiers

Rademacher complexity and generalization performance of multi-category margin classifiers

Rademacher complexity of margin multi-category classifiers

Structure discovery in PAC-learning by random projections

Contact Info

Product

Resources

About