An improved analysis of the Rademacher data-dependent bound using its self bounding property

Oneto, Luca; Ghio, Alessandro; Anguita, Davide; Ridella, Sandro

doi:10.1016/j.neunet.2013.03.017

Cited by 17 publications

(30 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(50). Thus, the previous bound is characterized (though in a non-typical optimistic scenario) by fast convergence: this is the first time for global RC measures, since they only showed slow convergence so far [14,16,55].…”

Section: Closed Form Boundsmentioning

confidence: 64%

“…refer to the discussion following Theorem 5.2 in [15]): for example, the first hypothesis does not hold when Gaussian kernels are used. Moreover, LRC bounds have also proved to be loose (as shown in this paper and already remarked by [55]), mostly because of the size of the constants which characterize them. Recently, because of the drawbacks of LRC bounds, some effort has been spent in order to leverage some of the basic ideas, driving LRC, in global RC bounds as well, targeted towards shrinking the hypothesis space and consequently reduce the overall impact of the complexity term [5][6][7].…”

Section: Introductionmentioning

confidence: 64%

“…The advantages and disadvantages of usingR n (H) orM n (H) are investigated in [10]: RC measuresR n (H) are preferred as they satisfy the hypothesis of [23] (refer to Theorem 2), with c = 2 n , and consequently we will focus on RC in this work. As shown in [14,55], it is also possible to improve the constants in Eq. (11) by removing the factor 3 in the best case scenarios (whenR n (H) → 0).…”

Section: Survey Of State-of-the-art Rademacher Complexity Boundsmentioning

confidence: 99%

“…It is now possible to show that the bound of Eq. (12) can be further improved by giving up a closed form expression [55], i.e. by formulating an implicit bound which requires a numerical procedure to be solved:…”

Section: Survey Of State-of-the-art Rademacher Complexity Boundsmentioning

confidence: 99%

“…the growth function and the Vapnik-Chervonenkis dimension, have been proposed in [70], and subsequently refined in [58] and [63]; however, data dependent alternatives, such as the Rademacher complexity (RC) [16,42,55], have proved to address the limitations of data independent measures. As RC is a global measure which contemplates the whole hypothesis space, improvements based on locality principles, namely local Rademacher complexity (LRC) bounds [14,15,43], have been proposed.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Global Rademacher Complexity Bounds: From Slow to Fast Convergence Rates

Oneto

Ghio

Ridella

et al. 2015

Neural Process Lett

Self Cite

View full text Add to dashboard Cite

Previousworksinliteratureshowedthatperformanceestimationsoflearningpro- cedures can be characterized by a convergence rate ranging between O(n^(−1)) (fast rate) and O(n^(−1/2)) (slow rate). In order to derive such result, some assumptions on the prob- lem are required; moreover, even when convergence is fast, the constants characterizing the bounds are often quite loose. In this work, we prove new Rademacher complexity (RC) based bounds, which do not require any additional assumptions for achieving a fast convergence rate O(n^(−1)) in the optimistic case and a slow rate O(n^(−1/2)) in the general case. At the same time, they are characterized by smaller constants with respect to other state-of-the-art RC fast converging alternatives in literature. The results proposed in this work are obtained by exploiting the fundamental work of Talagrand on concentration inequalities for product mea- sures and empirical processes. As a further issue, we also provide the extension of the results to the semi-supervised learning framework, showing how additional unlabeled samples allow improving the tightness of the derived bound

show abstract

Section: Closed Form Boundsmentioning

confidence: 64%

Section: Introductionmentioning

confidence: 64%

Section: Survey Of State-of-the-art Rademacher Complexity Boundsmentioning

confidence: 99%

Section: Survey Of State-of-the-art Rademacher Complexity Boundsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Global Rademacher Complexity Bounds: From Slow to Fast Convergence Rates

Oneto

Ghio

Ridella

et al. 2015

Neural Process Lett

Self Cite

View full text Add to dashboard Cite

show abstract

Model selection and error estimation without the agonizing pain

Oneto

2018

WIREs Data Min & Knowl

Self Cite

View full text Add to dashboard Cite

How can we select the best performing data-driven model? How can we rigorously estimate its generalization error? Statistical learning theory (SLT) answers these questions by deriving nonasymptotic bounds on the generalization error of a model or, in other words, by delivering upper bounding of the true error of the learned model based just on quantities computed on the available data. However, for a long time, SLT has been considered only as an abstract theoretical framework, useful for inspiring new learning approaches, but with limited applicability to practical problems. The purpose of this review is to give an intelligible overview of the problems of model selection (MS) and error estimation (EE), by focusing on the ideas behind the different SLT-based approaches and simplifying most of the technical aspects with the purpose of making them more accessible and usable in practice. We start by presenting the seminal works of the 80s until the most recent results, then discuss open problems and finally outline future directions of this field of research.

show abstract

The “Five W” of MS and EE

Oneto

2019

Model Selection and Error Estimation in a Nutshell

View full text Add to dashboard Cite

An improved analysis of the Rademacher data-dependent bound using its self bounding property

Cited by 17 publications

References 15 publications

Global Rademacher Complexity Bounds: From Slow to Fast Convergence Rates

Global Rademacher Complexity Bounds: From Slow to Fast Convergence Rates

Model selection and error estimation without the agonizing pain

The “Five W” of MS and EE

Contact Info

Product

Resources

About