Variable selection in reproducing kernel Hilbert space using random sketch method

Kang, Jongkyeong; Jhun, Myoungshic

doi:10.7465/jkdi.2020.31.4.501

Cited by 4 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar assumptions are also imposed in [19]. Assumption 2 assumes the boundedness of the kernel function and its gradient functions, and is satisfied by many popular kernels, including the Gaussian kernel and the Sobolev kernel [29,22,39] with the compact support condition. Note that the compact support condition is commonly used in machine learning literature [19,22,5,18] for mathematical simplicity, and it may be relaxed by allowing the support to expand with sample size, which leads to some additional treatment in the asymptotic analysis.…”

Section: Asymptotic Sparsistencymentioning

confidence: 99%

“…For simplicity, we denote these three methods as GM, DC-t and QaSIS-t, respectively. Note that the computational cost of most existing gradient-based methods [22,39]…”

Section: Numerical Experimentsmentioning

confidence: 99%

“…A novel measurement-error-model-based sparse learning method is developed in Stefanski et al. [30] and Wu and Stefanski [38] for nonparametric kernel regression models, and some gradient learning methods [22,39] are proposed to conduct sparse learning in a flexible RKHS [35]. Also, a flexible knock-off filter framework [1] and a recursive feature elimination method by using kernel ridge regression are proposed [5], which show substantial advantage than most existing methods, yet their lack of selection consistency or computational efficiency remain as some of their main obstacles.…”

Section: Introductionmentioning

confidence: 99%

“…Also, a flexible knock-off filter framework [1] and a recursive feature elimination method by using kernel ridge regression are proposed [5], which show substantial advantage than most existing methods, yet their lack of selection consistency or computational efficiency remain as some of their main obstacles. Specifically, it is also interesting to point out that most existing gradient-based methods [22,39] aim to directly estimate the gradient functions in a regularization framework with some well-designed penalty terms, and thus they may not be applicable to analyze data with high dimension due to their expensive computational cost.…”

Section: Introductionmentioning

confidence: 99%

“…Unlike most nonparametric models, we measure the significance of each gradient function to distinguish the informative and uninformative variables without any explicit model specification. Note that the required minimal signal strength in Assumption 4 is much tighter than that in many nonparametric sparse learning methods[14,39], which often require the signal to be bounded away from zero. Now we establish the asymptotic sparsistency of the proposed sparse learning method.Theorem 2.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Efficient kernel-based variable selection with sparsistency

He¹,

Wang²,

Lv³

2021

STAT SINICA

View full text Add to dashboard Cite

Sparse learning is central to high-dimensional data analysis, and various methods have been developed. Ideally, a sparse learning method shall be methodologically flexible, computationally efficient, and with theoretical guarantee, yet most existing methods need to compromise some of these properties to attain the other ones. In this article, a three-step sparse learning method is developed, involving kernel-based estimation of the regression function and its gradient functions as well as a hard thresholding. Its key advantage is that it assumes no explicit model assumption, admits general predictor effects, allows for efficient computation, and attains desirable asymptotic sparsistency. The proposed method can be adapted to any reproducing kernel Hilbert space (RKHS) with different kernel functions, and its computational cost is only linear in the data dimension. The asymptotic sparsistency of the proposed method is established for general RKHS under mild conditions. Numerical experiments also support that the proposed method compares favorably against its competitors in both simulated and real examples.

show abstract

Section: Asymptotic Sparsistencymentioning

confidence: 99%

“…For simplicity, we denote these three methods as GM, DC-t and QaSIS-t, respectively. Note that the computational cost of most existing gradient-based methods [22,39]…”

Section: Numerical Experimentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Efficient kernel-based variable selection with sparsistency

He¹,

Wang²,

Lv³

2021

STAT SINICA

View full text Add to dashboard Cite

show abstract

Fair Kernel Regression via Fair Feature Embedding in Kernel Space

Okray

Lan

2019

2019 IEEE 31st International Conference on Tools With Artificial Intelligence (ICTAI)

View full text Add to dashboard Cite

In recent years, there have been significant efforts on mitigating unethical demographic biases in machine learning methods. However, very little work is done for kernel methods. In this paper, we propose a novel fair kernel regression method via fair feature embedding (FKR-F 2 E) in kernel space. Motivated by prior works feature processing for fair learning and feature selection for kernel methods, we propose to learn fair feature embeddings in kernel space, where the demographic discrepancy of feature distributions is minimized. Through experiments on three public real-world data sets, we show the proposed FKR-F 2 E achieves significantly lower prediction disparity compared with the state-of-the-art fair kernel regression method and several other baseline methods.

show abstract

Learning sparse conditional distribution: An efficient kernel-based approach

Chen

Wang

2021

Electron. J. Statist.

View full text Add to dashboard Cite

This paper proposes a novel method to recover the sparse structure of the conditional distribution, which plays a crucial role in subsequent statistical analysis such as prediction, forecasting, conditional distribution estimation and others. Unlike most existing methods that often require explicit model assumption or suffer from computational burden, the proposed method shows great advantage by making use of some desirable properties of reproducing kernel Hilbert space (RKHS). It can be efficiently implemented by optimizing its dual form and is particularly attractive in dealing with large-scale dataset. The asymptotic consistencies of the proposed method are established under mild conditions. Its effectiveness is also supported by a variety of simulated examples and a real-life supermarket dataset from Northern China.

show abstract

Variable selection in reproducing kernel Hilbert space using random sketch method

Cited by 4 publications

References 0 publications

Efficient kernel-based variable selection with sparsistency

Efficient kernel-based variable selection with sparsistency

Fair Kernel Regression via Fair Feature Embedding in Kernel Space

Learning sparse conditional distribution: An efficient kernel-based approach

Contact Info

Product

Resources

About