A component lasso

Abstract

We propose a new sparse regression method called the component lasso, based on a simple idea. The method uses the connected‐components structure of the sample covariance matrix to split the problem into smaller ones. It then applies the lasso to each subproblem separately, obtaining a coefficient vector for each one. Finally, it uses non‐negative least squares to recombine the different vectors into a single solution. This step is useful in selecting and reweighting components that are correlated with the response. We prove that the component lasso is strongly sign consistent in a block‐diagonal setting. Simulated and real data examples show that the component lasso can outperform standard regression methods such as the lasso and elastic net, achieving a lower mean squared error as well as better support recovery. The modular structure of the algorithm also lends itself naturally to parallel computation. The Canadian Journal of Statistics 43: 624–646; 2015 © 2015 Statistical Society of Canada

Comparison of Penalized Regression Methods through a Simulation Study

“…Değerlendirme yöntemleri regresyon modelinin ön tahmin performansını değerlendirerek ayar parametresi tahmininde kullanılan yöntemlerdir. Bir diğer teknik veri kümesinin eğitim değerlendirme ve test kümelerine ayrıştırılması ilkesine dayanır [10,22]. Bu teknikte cezalı regresyon yöntemleri tüm ayar parametresi değerleri için eğitime tabi tutulur.…”

A note on moment inequality for quadratic forms

“…Since the precision matrix fully encodes the partial correlation graph [9], (7) naturally rises in the problems of testing the graph structures of a large number of variables; for instance, different block structures on the pathway and modularity in the genetic regulatory networks; c.f. [17,8]. When x ∼ N(0, Ω −1 ), we use the (log-)likelihood ratio test (LRT) statistic…”

Sparse estimation: applications in atrial fibrillation

“…The lasso algorithm has been extended to work for grouped variables called the group lasso [104] and sparse-group lasso [90]. Component lasso [46] is another extension that groups variables based on the sample covariance matrix. Sparse approaches have been developed for the class of generalized linear models, to include for instance sparse logistic regression and Cox's proportional hazards analysis [89].…”