Optimal selection of reduced rank estimators of high-dimensional matrices

Bunea, Florentina; She, Yiyuan; Wegkamp, Marten

doi:10.1214/11-aos876

Cited by 206 publications

(313 citation statements)

References 21 publications

Supporting

Mentioning

300

Contrasting

Unclassified

Order By: Relevance

“…Since minimizing the rank function (7) is generally intractable, and the matrix nuclear norm is usually used as a tight convex surrogate of the matrix rank [35], the rank function (7) can be replaced with the tensor nuclear norm [27]:…”

Section: Nonlocal Low-rank Tensor Approximationmentioning

confidence: 99%

“…σ r (Z) is so-called the nuclear norm of matrix Z with size m × n. Although a convex tensor nuclear norm, such as (8), could provide satisfactory results in various tensor recovery problems, studies like [35] demonstrated that the matrix nuclear norm over-penalizes large singular values, and thus gives a biased estimator in low-rank structure learning. Fortunately, a folded-concave penalty [26] can be considered to remedy such modeling bias [25,26].…”

Section: Nonlocal Low-rank Tensor Approximationmentioning

confidence: 99%

See 1 more Smart Citation

Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization

et al. 2017

View full text Add to dashboard Cite

Hyperspectral image (HSI) possesses three intrinsic characteristics: the global correlation across spectral domain, the nonlocal self-similarity across spatial domain, and the local smooth structure across both spatial and spectral domains. This paper proposes a novel tensor based approach to handle the problem of HSI spatial super-resolution by modeling such three underlying characteristics. Specifically, a noncovex tensor penalty is used to exploit the former two intrinsic characteristics hidden in several 4D tensors formed by nonlocal similar patches within the 3D HSI. In addition, the local smoothness in both spatial and spectral modes of the HSI cube is characterized by a 3D total variation (TV) term. Then, we develop an effective algorithm for solving the resulting optimization by using the local linear approximation (LLA) strategy and the alternative direction method of multipliers (ADMM). A series of experiments are carried out to illustrate the superiority of the proposed approach over some state-of-the-art approaches.

show abstract

Section: Nonlocal Low-rank Tensor Approximationmentioning

confidence: 99%

Section: Nonlocal Low-rank Tensor Approximationmentioning

confidence: 99%

Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The proposed algorithm requires an initial estimate, U ð0Þ , V ð0Þ , and d ð0Þ , which can be obtained from an initial estimate of C through the SVD. One plausible estimator is the reduced-rank least squares estimator (33,34), which is consistent for high-dimensional data (35). Another one is the ridge regression estimator in which a small identity matrix «I p is added to Σ to make it invertible.…”

Section: T-svd Model With the Svd Representation Of The Coefficient mentioning

confidence: 99%

Learning regulatory programs by threshold SVD regression

Ma¹,

Xiao

Wong

2014

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

We formulate a statistical model for the regulation of global gene expression by multiple regulatory programs and propose a thresholding singular value decomposition (T-SVD) regression method for learning such a model from data. Extensive simulations demonstrate that this method offers improved computational speed and higher sensitivity and specificity over competing approaches. The method is used to analyze microRNA (miRNA) and long noncoding RNA (lncRNA) data from The Cancer Genome Atlas (TCGA) consortium. The analysis yields previously unidentified insights into the combinatorial regulation of gene expression by noncoding RNAs, as well as findings that are supported by evidence from the literature.regulatory program | SVD | sparse | multivariate | regression T he development of microarray and next-generation sequencing technologies has enabled rapid quantification of various genome-wide features (DNA sequences, gene expressions, noncoding RNA expressions, methylation, etc.) in a population of samples (1, 2). Large consortia have compiled genetic and molecular profiling data in an enormous number of tumors across hundreds of samples (3,4). A common challenge arising from these large-scale genomic studies is the inference of regulatory relationships between different genome-wide measurements from the complex biological systems where the number of predictors and responses often far exceeds the sample size.To formulate a statistical model for such regulatory relations, consider the situation depicted in Fig. 1 (see Fig. S1 for more detailed illustration of the model schema), where p regulators x = ðx 1 ; . . . ; x p Þ regulate q responses y = ðy 1 ; . . . ; y q Þ through r regulatory programs that are represented by hidden nodes, e.g., h 1 ; . . . ; h r . The activity h j of the jth program depends on the regulators connected to hidden node j, and h j in turn affects the level of the responses that are connected to node j. To express this model mathematically, we denote by u j and v j the unit vectors corresponding respectively to the input weights fa ij , i = 1; . . . ; pg and the output weights fb jk , k = 1; . . . ; qg of the jth program. Then the regulatory relations are represented as h j = σðxu j Þ and y = P r j=1 d j h j v′ j , where x, y are regarded as row vectors, u, v are regarded as column vectors, and σ() is a sigmoidal function. The aforementioned is a standard single-layer neural network model that is widely used in predictive modeling but could be impossible to learn in biological studies where sample size n is much smaller than p or q. Thus, we first simplify the model by taking σ to be the identity function. Then our model becomes h j = xu j , y = P r j=1 d j h j v′ j . We make the biologically plausible assumption that only a small subset of regulators is contributing to any program and that each program regulates only a small subset of responses. Under this assumption, u j and v j are sparse vectors in R p and R q , respectively. The magnitude of the output weight vector (denoted by d j ) repr...

show abstract

“…Wenn die Faktoren unbekannt sind, kann zur Schätzung und Bestimmung des Modells die reduced-rank multivariate Regression angewendet werden, bei der sowohl die Ziel-, als auch die Eingangsgrößenüber eine Matrix gekoppelte Vektoren sind. Yuan et al (2007), Negahban and Wainwright (2011) and Bunea et al (2011) zeigen, dass unter Einsatz der Ky-Fan-Norm oder Rang Regularisierung die Anzahl der Faktoren mit hoher Wahrscheinlichkeit geschätzt werden kann. Allerdings konzentrieren sich die bisher untersuchten Modelle nur auf bedingte Erwartungswerte und geben wenig Informationenüber die bedingten Verteilungen.…”

Section: IIIunclassified

“…One remark is that for the traditional multivariate regression technique introduced in Reinsel and Velu (1998), the number of factor r is assumed to be known or has to be obtained via other method. However, using the modern regularization method of Yuan et al (2007), Bunea et al (2011) or Negahban and Wainwright (2011), knowing r is not necessary for estimation.…”

Section: Factorizable Sparse Multivariate Quantile Regressionmentioning

confidence: 99%

Quantile Regression in Risk Calibration

Chao

Härdle

Wang

2014

Handbook of Financial Econometrics and Statistics

View full text Add to dashboard Cite

Quantile regression studies the conditional quantile function Q Y |X (τ ) on X at level τ which satisfies F Y |X  Q Y |X (τ )  = τ , where F Y |X is the conditional CDF of Y given X, ∀τ ∈ (0, 1). Quantile regression allows for a closer inspection of the conditional distribution beyond the conditional moments. This technique is particularly useful in, for example, the Value-at-Risk (VaR) which the Basel accords (2011) require all banks to report, or the "quantile treatment effect" and "conditional stochastic dominance (CSD)" which are economic concepts in measuring the effectiveness of a government policy or a medical treatment.Given its value of applicability, to develop the technique of quantile regression is, however, more challenging than mean regression. It is necessary to be adept with general regression problems and M -estimators; additionally one needs to deal with non-smooth loss functions. In this dissertation, chapter 2 is devoted to empirical risk management during financial crises using quantile regression. Chapter 3 and 4 address the issue of high-dimensionality and the nonparametric technique of quantile regression.Chapter 2 applies nonparametric confidence bands for quantile functions to investigate the tail dependence of stock returns. It is shown that strong nonlinear correlation exists when stock prices drop, confirming the fact that in financial crises, firms are more dependent on each other than when the market is booming. This sheds light on the risk management of counterparty risk.In Chapter 3, motivated by applications in economics like quantile treatment effects, or conditional stochastic dominance, we focus on the construction of confidence corridors for nonparametric multivariate kernel quantile and expectile regression functions. Through an uniform kernel Bahadur representation for M -estimators, strong Gaussian approximation and asymptotic extreme value theory we derive the asymptotic confidence corridor for the nonparametric kernel conditional quantile/expectile functions. We find that the bands for quantile/expectile functions are wide when τ is close to 0 and 1 due to the variance of the estimator. The coverage ratios given by the asymptotic confidence corridors are meager. To deal with this issue, we propose a novel smoothing bootstrap which gives satisfactory coverage ratios while keeping the size of the confidence corridors in a reasonable range. Our method contributes to the differentiation between the "risk reduction CSD" and "potential enhancement CSD", which is not possible by using techniques based on previous research in CSD like Delgado and Escanciano (2013). This differentiation is crucial as the two types of CSD may induce different utility to the government and citizens. After applying our method to the data set from National Supported Work Demonstration, a temporary internship program offered to disadvantaged workers, it is found that this program tends to be "potential enhancement CSD" and it may not help foster the employment of less capable people as much...

show abstract

Optimal selection of reduced rank estimators of high-dimensional matrices

Cited by 206 publications

References 21 publications

Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization

Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization

Learning regulatory programs by threshold SVD regression

Quantile Regression in Risk Calibration

Contact Info

Product

Resources

About