On consistency and sparsity for sliced inverse regression in high dimensions

Lin, Qian; Zhao, Zhigen; Liu, Jun S.

doi:10.48550/arxiv.1507.03895

Cited by 10 publications

(32 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we first prove a conjecture regarding the coordinate-wise sliced stability conditions introduced by Lin et al (2015). Second, we establish the optimal rate of support recovery in terms of the sample size.…”

Section: Resultsmentioning

confidence: 96%

“…Relying on the subsequent developments of this section, in Example 2 and Remark 3 of Appendix B we demonstrate that models of the form Y = G(h(X β) + ε), where G, h are continuous and monotone and ε is a log-concave random variable, satisfy the sliced stability assumption. Y Definition 1 is the sliced stability definition from Lin et al (2015) restated in terms of the SIM. Lin et al (2015) conjectured that the sliced stability condition could be implied from the well accepted conditions proposed by Hsing & Carroll (1992), which we state below with a slight modification.…”

Section: Sliced Stabilitymentioning

confidence: 99%

“…Sliced inverse regression, proposed by Li (1991), is one of the most popular SDR methods for estimating the space S. When the dimensionality p is larger than or comparable to the sample size n, sparsity assumptions are often imposed on the loading vector β (Li & Nachtsheim 2006, e.g.). Lin et al (2015) proved that in fact E[∠( β, β)] > 0 if ρ = lim p n = 0 and sin ∠( β, β) = 0 when ρ = 0 where β is the SIR estimator of β. In other words, the SIR estimator β is consistent (up to a sign) if and only if ρ = lim p n = 0.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we study two procedures for signed support recover of SIM (1): the DT-SIR introduced by Lin et al (2015) and the SDP approach inspired by Amini & Wainwright (2008). We let Γ = n s log(p−s) be the rescaled sample size.…”

Section: Introductionmentioning

confidence: 99%

“…To the best of our knowledge, this optimality result, regarding the sample size of SIM (1), has not been previously discussed in the literature. Our second contribution is, to establish a sliced stability conjecture formulated by Lin et al (2015), under the SIM case. We demonstrate that classical conditions of Hsing & Carroll (1992) imply sliced stability in Section 2.1.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Signed support recovery for single index models in high-dimensions

Neykov

Lin

Liu

2016

Annals of Mathematical Sciences and Applications

Self Cite

View full text Add to dashboard Cite

In this paper we study the support recovery problem for single index models Y = f (X β, ε), where f is an unknown link function, X ∼ N p (0, I p ) and β is an s-sparse unit vector such that β i ∈ {± 1 √ s , 0}. In particular, we look into the performance of two computationally inexpensive algorithms: (a) the diagonal thresholding sliced inverse regression (DT-SIR) introduced by Lin et al. (2015); and (b) a semi-definite programming (SDP) approach inspired by Amini & Wainwright (2008). When s = O(p 1−δ ) for some δ > 0, we demonstrate that both procedures can succeed in recovering the support of β as long as the rescaled sample size Γ = n s log(p−s) is larger than a certain critical threshold. On the other hand, when Γ is smaller than a critical value, any algorithm fails to recover the support with probability at least 1 2 asymptotically. In other words, we demonstrate that both DT-SIR and the SDP approach are optimal (up to a scalar) for recovering the support of β in terms of sample size. We provide extensive simulations, as well as a real dataset application to help verify our theoretical observations.

show abstract

Section: Resultsmentioning

confidence: 96%

Section: Sliced Stabilitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Signed support recovery for single index models in high-dimensions

Neykov

Lin

Liu

2016

Annals of Mathematical Sciences and Applications

Self Cite

View full text Add to dashboard Cite

show abstract

Misspecified nonconvex statistical optimization for sparse phase retrieval

Yang

Fang

et al. 2019

Math. Program.

View full text Add to dashboard Cite

Existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying "true" statistical models. To address this issue, we take a first step towards taming model misspecification by studying the high-dimensional sparse phase retrieval problem with misspecified link functions. In particular, we propose a simple variant of the thresholded Wirtinger flow algorithm that, given a proper initialization, linearly converges to an estimator with optimal statistical accuracy for a broad family of unknown link functions. We further provide extensive numerical experiments to support our theoretical findings.where β 0 denotes the number of nonzero entries in β.The nonconvex problem in (1.1) gives rise to two challenges in optimization and statistics. From the perspective of optimization, (1.1) is NP-hard in the worst case (Sahinoglou and Cabrera, 1991), Equal contribution. . 1 Here we use the shorthand [n] = {1, 2, . . . , n}. 1 arXiv:1712.06245v1 [stat.ML] 18 Dec 2017that is, under computational hardness hypotheses, no algorithm can achieve the global minimum in polynomial time. Particularly, most existing general-purpose first-order or second-order optimization methods (Ghadimi and Lan, 2013;Bolte et al., 2014;Lu and Xiao, 2015;Hong et al., 2016;Ghadimi and Lan, 2016;Xu and Yin, 2017;Gonçalves et al., 2017) are only guaranteed to converge to certain stationary points. Meanwhile, since (1.1) can also be cast as a polynomial optimization problem, we can leverage various semidefinite programming approaches (Parrilo, 2003;Kim et al., 2016;Weisser et al., 2017;Ahmadi and Parrilo, 2017). However, in real applications the problem dimension of practical interest is often large, for example, p can be of the order of millions. To the best of our knowledge, existing polynomial optimization approaches do not scale up to such large dimensions. The difficulty in optimization further leads to more challenges in statistics. From the perspective of statistics, researchers are interested in characterizing the statistical properties of β with respect to some underlying ground truth β * , for example, the estimation error β − β * 2 . Nevertheless, due to the lack of global optimality in nonconvex optimization, the statistical properties of the solutions obtained by existing algorithms remain rather difficult to analyze.Recently, Cai et al. (2016) proposed a thresholded Wirtinger flow (TWF) algorithm to tackle the problem, which essentially employs proximal-type iterations. TWF starts from a carefully specified initial point, and iteratively performs gradient descent steps. In particular, at each iteration, TWF performs a thresholding step to preserve the sparsity of the solution. Cai et al. (2016) further prove that TWF achieves a linear rate of convergence to an approximate global minimum that has optimal statistical accuracy. Note that Cai et al. (2016) can establish such strong theoretical results because their algorithm and analysis exploit the underlying "true" data generating process...

show abstract

Optimal estimation of slope vector in high-dimensional linear transformation models

Tan

2019

Journal of Multivariate Analysis

View full text Add to dashboard Cite

On consistency and sparsity for sliced inverse regression in high dimensions

Cited by 10 publications

References 20 publications

Signed support recovery for single index models in high-dimensions

Signed support recovery for single index models in high-dimensions

Misspecified nonconvex statistical optimization for sparse phase retrieval

Optimal estimation of slope vector in high-dimensional linear transformation models

Contact Info

Product

Resources

About