Prediction bounds for higher order total variation regularized least squares

Ortelli, Francesco; Geer, Sara van de

doi:10.48550/arxiv.1904.10871

Cited by 5 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can now use the existing bound given by Theorem 1.1 in Ortelli and van de Geer (2019) to bound SSE( θ (λ * ) , θ * ) for the usual Trend Filtering estimator by generalizing its proof to hold for general subgaussian errors. Therefore, the desired bound for min λ∈Λ SSE( θ (λ) , θ * ) again follows pretty much directly from the existing result Theorem 1.1 in Ortelli and van de Geer (2019).…”

Section: Sketch Of Trend Filtering Proofsmentioning

confidence: 73%

“…4. We only state Theorem 4.2 for r ∈ {1, 2, 3, 4} and the assumptions on θ * in Theorem 4.2 are identical to the assumptions made in Theorem 1.1 of Ortelli and van de Geer (2019). This is because our proof is based on the proof technique employed by Ortelli and van de Geer ( 2019), as explained in Section 12.1.…”

Section: Main Results For the Cvtf Estimatormentioning

confidence: 99%

“…In view of Theorem 2.1, it is enough to bound min λ∈Λ SSE( θ State of the art existing bounds for penalized Trend Filtering are available in Ortelli and van de Geer (2019). In particular, Theorem 1.1 in Ortelli and van de Geer (2019) gives the desired upper bounds (both slow and fast rates) on SSE( θ…”

Section: Sketch Of Trend Filtering Proofsmentioning

confidence: 99%

“…In order to prove Proposition 12.2, we again bound T 3 , but this time in a different manner. This is to prove the fast rate theorem and this part uses the proof technique via interpolating vectors pioneered by Ortelli and van de Geer (2019).…”

Section: Basic Inequalitymentioning

confidence: 99%

“…In contrast to classical nonparametric regression methods such as local polynomials, splines, kernels etc., TF is a spatially adaptive method as the knots of the piecewise polynomials are chosen in a data driven fashion. The last few years have seen a flurry of research (e.g Tibshirani (2014), Guntuboyina et al (2020), Ortelli and van de Geer (2019)) in trying to understand the theoretical properties of TF. However, all the existing guarantees hold when the tuning parameter is chosen in an optimal way depending on problem parameters which are typically unknown.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

Chaudhuri¹,

Chatterjee²

2022

Preprint

View full text Add to dashboard Cite

This paper formulates a general cross validation framework for signal denoising. The general framework is then applied to nonparametric regression methods such as Trend Filtering and Dyadic CART. The resulting cross validated versions are then shown to attain nearly the same rates of convergence as are known for the optimally tuned analogues. There did not exist any previous theoretical analyses of cross validated versions of Trend Filtering or Dyadic CART. To illustrate the generality of the framework we also propose and study cross validated versions of two fundamental estimators; lasso for high dimensional linear regression and singular value thresholding for matrix estimation. Our general framework is inspired by the ideas in Chatterjee and Jafarov (2015) and is potentially applicable to a wide range of estimation methods which use tuning parameters.

show abstract

Section: Sketch Of Trend Filtering Proofsmentioning

confidence: 73%

Section: Main Results For the Cvtf Estimatormentioning

confidence: 99%

Section: Sketch Of Trend Filtering Proofsmentioning

confidence: 99%

Section: Basic Inequalitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

Chaudhuri¹,

Chatterjee²

2022

Preprint

View full text Add to dashboard Cite

show abstract

Logistic regression with total variation regularization

van de Geer

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We study logistic regression with total variation penalty on the canonical parameter and show that the resulting estimator satisfies a sharp oracle inequality: the excess risk of the estimator is adaptive to the number of jumps of the underlying signal or an approximation thereof. In particular when there are finitely many jumps, and jumps up are sufficiently separated from jumps down, then the estimator converges with a parametric rate up to a logarithmic term log n/n, provided the tuning parameter is chosen appropriately of order 1/ √ n. Our results extend earlier results for quadratic loss to logistic loss. We do not assume any a priori known bounds on the canonical parameter but instead only make use of the local curvature of the theoretical risk.

show abstract

Optimal Dynamic Regret in Exp-Concave Online Learning

Baby,

Wang

2021

Preprint

View full text Add to dashboard Cite

We consider the problem of the Zinkevich (2003)-style dynamic regret minimization in online learning with exp-concave losses. We show that whenever improper learning is allowed, a Strongly Adaptive online learner achieves the dynamic regret of Õ(dwhere Cn is the total variation (a.k.a. path length) of the an arbitrary sequence of comparators that may not be known to the learner ahead of time. Achieving this rate was highly nontrivial even for squared losses in 1D where the best known upper bound was O( √ nCn ∨ log n) (Yuan and Lamperski, 2019). Our new proof techniques make elegant use of the intricate structures of the primal and dual variables imposed by the KKT conditions and could be of independent interest. Finally, we apply our results to the classical statistical problem of locally adaptive non-parametric regression (Mammen, 1991;Donoho and Johnstone, 1998) and obtain a stronger and more flexible algorithm that do not require any statistical assumptions or any hyperparameter tuning.

show abstract

Prediction bounds for higher order total variation regularized least squares

Cited by 5 publications

References 16 publications

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

Logistic regression with total variation regularization

Optimal Dynamic Regret in Exp-Concave Online Learning

Contact Info

Product

Resources

About