MARS via LASSO

Ki, Dohyeong; Fang, Billy; Guntuboyina, Adityanand

doi:10.48550/arxiv.2111.11694

Cited by 3 publications

(3 citation statements)

References 31 publications

(135 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our CV framework is based on a general principle and should be looked upon as providing a general recipe to develop theoretically tractable CV versions of potentially any other estimator which uses a tuning parameter. For example, using our framework, one should be able to develop theoretically tractable CV versions of the Total Variation Denoising estimator proposed by Rudin et al (1992) (also see Hütter and Rigollet (2016), Sadhanala et al (2016), Chatterjee and Goswami (2019b)), the Hardy Krauss estimator (see Fang et al (2021), Ortelli and van de Geer (2020)), the Optimal Regression Tree estimator proposed in Chatterjee and Goswami (2019a), a higher dimensional version of Trend Filtering of order 2 proposed in Ki et al (2021) and potentially many more. As a start, the Zero Doubling method of constructing completion estimators alongwith using a geometrically doubling grid of candidate tuning values should be useable in these problems.…”

Section: Other Signal Denoising Methodsmentioning

confidence: 99%

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

Chaudhuri¹,

Chatterjee²

2022

Preprint

View full text Add to dashboard Cite

This paper formulates a general cross validation framework for signal denoising. The general framework is then applied to nonparametric regression methods such as Trend Filtering and Dyadic CART. The resulting cross validated versions are then shown to attain nearly the same rates of convergence as are known for the optimally tuned analogues. There did not exist any previous theoretical analyses of cross validated versions of Trend Filtering or Dyadic CART. To illustrate the generality of the framework we also propose and study cross validated versions of two fundamental estimators; lasso for high dimensional linear regression and singular value thresholding for matrix estimation. Our general framework is inspired by the ideas in Chatterjee and Jafarov (2015) and is potentially applicable to a wide range of estimation methods which use tuning parameters.

show abstract

Section: Other Signal Denoising Methodsmentioning

confidence: 99%

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

Chaudhuri¹,

Chatterjee²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Other models for multivariate TV. There has been a recent surge of work studying different generalizations of TV and trend filtering penalties to multiple dimensions, for example, Bibaut and van der Laan (2019); ; Ortelli and van de Geer (2021); Chatterjee and Goswami (2021); Ki et al (2021). Many of these works are based on the notion of Hardy-Krause variation of a multivariate function, or the related notion of Vitali variation.…”

Section: Continuous-time Multivariate Tv Methodsmentioning

confidence: 99%

Multivariate Trend Filtering for Lattice Data

Sadhanala¹,

Wang²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

We study a multivariate version of trend filtering, called Kronecker trend filtering or KTF, for the case in which the design points form a lattice in d dimensions. KTF is a natural extension of univariate trend filtering (Steidl et al., 2006;Kim et al., 2009;Tibshirani, 2014), and is defined by minimizing a penalized least squares problem whose penalty term sums the absolute (higher-order) differences of the parameter to be estimated along each of the coordinate directions. The corresponding penalty operator can be written in terms of Kronecker products of univariate trend filtering penalty operators, hence the name Kronecker trend filtering. Equivalently, one can view KTF in terms of an 1-penalized basis regression problem where the basis functions are tensor products of falling factorial functions, a piecewise polynomial (discrete spline) basis that underlies univariate trend filtering.This paper is a unification and extension of the results in Sadhanala et al. (2016, 2017). We develop a complete set of theoretical results that describe the behavior of k th order Kronecker trend filtering in d dimensions, for every k ≥ 0 and d ≥ 1. This reveals a number of interesting phenomena, including the dominance of KTF over linear smoothers in estimating heterogeneously smooth functions, and a phase transition at d = 2(k + 1), a boundary past which (on the high dimension-to-smoothness side) linear smoothers fail to be consistent entirely. We also leverage recent results on discrete splines from Tibshirani (2020), in particular, discrete spline interpolation results that enable us to extend the KTF estimate to any off-lattice location in constant-time (independent of the size of the lattice n).

show abstract

“…However, for many function classes, this kind of approximability may not hold. For example, we can consider the class of Hardy Krause Bounded Variation Functions (see Fang et al (2021)) or its higher order versions (see Ki et al (2021)) where the existing covering argument produces nets (to estimate metric entropy) which are not necessarily rectangular piecewise constant/linear respectively. These function classes are also known not to suffer from the curse of dimensionality in the sense that the metric entropy does not grow exponentially in 1 with the dimension d. More generally, it would be very interesting to come up with computationally efficient and statistically rate optimal online prediction algorithms for such function classes.…”

Section: 2mentioning

confidence: 99%

Spatially Adaptive Online Prediction of Piecewise Regular Functions

Chatterjee¹,

Goswami²

2022

Preprint

View full text Add to dashboard Cite

We consider the problem of estimating piecewise regular functions in an online setting, i.e., the data arrive sequentially and at any round our task is to predict the value of the true function at the next revealed point using the available data from past predictions. We propose a suitably modified version of a recently developed online learning algorithm called the sleeping experts aggregation algorithm. We show that this estimator satisfies oracle risk bounds simultaneously for all local regions of the domain. As concrete instantiations of the expert aggregation algorithm proposed here, we study an online mean aggregation and an online linear regression aggregation algorithm where experts correspond to the set of dyadic subrectangles of the domain. The resulting algorithms are near linear time computable in the sample size. We specifically focus on the performance of these online algorithms in the context of estimating piecewise polynomial and bounded variation function classes in the fixed design setup. The simultaneous oracle risk bounds we obtain for these estimators in this context provide new and improved (in certain aspects) guarantees even in the batch setting and are not available for the state of the art batch learning estimators.

show abstract

MARS via LASSO

Cited by 3 publications

References 31 publications

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

A Cross Validation Framework for Signal Denoising with Applications to Trend Filtering, Dyadic CART and Beyond

Multivariate Trend Filtering for Lattice Data

Spatially Adaptive Online Prediction of Piecewise Regular Functions

Contact Info

Product

Resources

About