Faster convergence rate for functional linear regression in reproducing kernel Hilbert spaces

Zhang, Fode; Zhang, Weiping; Li, Rui; Lian, Heng

doi:10.1080/02331888.2019.1694931

Cited by 7 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since it aggressively targets the reduction of residual errors, it is commonly observed in practical applications that the CG method achieves convergence in signicantly fewer iterations compared to other gradient descent techniques, as discussed in the context of kernel learning by [13] and [8]. We obtain a convergence rate for β − β * L 2 (S) and show it to align with the minimax rates of the FLR model [6,27], thereby establishing the minimax optimality of our estimator.…”

Section: Introductionsupporting

confidence: 54%

“…The analysis depends on the eigenvalue behaviour of the operator Λ = T Note that the assumption implies β * ∈ H with additional smoothness. In [27], the authors use this source condition to derive the minimax and faster convergence rate for the Tikhonov regularization with 0 < α ≤ 1 2 .…”

Section: The Main Resultsmentioning

confidence: 99%

“…Later in [6], they used the regularization technique to achieve optimal rates without the Sacks Ylvisaker condition, which was a necessary assumption in [26]. Further analysis of the FLR model within the framework of RKHS has been studied and discussed in [2,24,25,27]. Since the computational complexity of these techniques is O(n 3 ), they incur high computational costs when dealing with large datasets.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Convergence analysis of kernel conjugate gradient for functional linear regression

Gupta,

Sivananthan,

Sriperumbudur

2023

J. App. Num. Analysis

View full text Add to dashboard Cite

In this paper, we discuss the convergence analysis of the conjugate gradient-based algorithm for the functional linear model in the reproducing kernel Hilbert space framework, utilizing early stopping results in regularization against over-fitting. We establish the convergence rates depending on the regularity condition of the slope function and the decay rate of the eigenvalues of the operator composition of covariance and kernel operator. Our convergence rates match the minimax rate available from the literature.

show abstract

Section: Introductionsupporting

confidence: 54%

Section: The Main Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence analysis of kernel conjugate gradient for functional linear regression

Gupta,

Sivananthan,

Sriperumbudur

2023

J. App. Num. Analysis

View full text Add to dashboard Cite

show abstract

“…A6* is more interpretable and easy to verify than A6. Its is also more commonly used in literature (Brownlees et al, 2015;Zhang et al, 2020;Paul et al, 2021b).…”

Section: Alternative Assumptionsmentioning

confidence: 99%

“…Linear models, especially regularized ones are well known to admit faster rates of convergence than the ERM rate of O(1/ √ n). For example, SVMs (Steinwart and Scovel, 2007), linear models (Sridharan et al, 2008) and more recently functional linear models (Zhang et al, 2020) are all known to achieve an error rate of O(1/n). It should come as no surprise that MoM estimators also achieve this so-called "fast rate" under additional assumptions.…”

Section: Fast Ratesmentioning

confidence: 99%

Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

Chakraborty¹,

Paul²,

Das³

2022

Preprint

View full text Add to dashboard Cite

The problem of linear predictions has been extensively studied for the past century under pretty generalized frameworks. Recent advances in the robust statistics literature allow us to analyze robust versions of classical linear models through the prism of Median of Means (MoM). Combining these approaches in a piecemeal way might lead to ad-hoc procedures, and the restricted theoretical conclusions that underpin each individual contribution may no longer be valid. To meet these challenges coherently, in this study, we offer a unified robust framework that includes a broad variety of linear prediction problems on a Hilbert space, coupled with a generic class of loss functions. Notably, we do not require any assumptions on the distribution of the outlying data points (O) nor the compactness of the support of the inlying ones (I). Under mild conditions on the dual norm, we show that for misspecification level ǫ, these estimators achieve an error rate of O(max |O| 1/2 n −1/2 , |I| 1/2 n −1 + ǫ), matching the best-known rates in literature. This rate is slightly slower than the classical rates of O(n −1/2 ), indicating that we need to pay a price in terms of error rates to obtain robust estimates. Additionally, we show that this rate can be improved to achieve so-called "fast rates" under additional assumptions.

show abstract

Additive functional regression in reproducing kernel Hilbert spaces under smoothness condition

Tian

Lin

Lian

et al. 2020

Metrika

View full text Add to dashboard Cite

Faster convergence rate for functional linear regression in reproducing kernel Hilbert spaces

Cited by 7 publications

References 18 publications

Convergence analysis of kernel conjugate gradient for functional linear regression

Convergence analysis of kernel conjugate gradient for functional linear regression

Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

Additive functional regression in reproducing kernel Hilbert spaces under smoothness condition

Contact Info

Product

Resources

About