Majorization minimization by coordinate descent for concave penalized generalized linear models

Jiang, Dongjie; Huang, Jian

doi:10.1007/s11222-013-9407-3

Cited by 12 publications

(17 citation statements)

References 24 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the notation

{\overset{italicβ}{true}}^{false(k, m false)}

to keep track of the ( m th) “inner” CD iteration, defined within each ( k th) step of Equation . Jiang and Huang () developed an efficient implementation of CD, termed the majorization minimization by CD (MMCD), which we employ in this paper to obtain Equation , for each ( k th) “outer” loop step. CD cyclically updates each ( j th) coordinate while holding the other coordinates fixed, until convergence of its iteration:

{\overset{italicβ}{true}}_{j}^{false(k, m false)} = {false({\overset{β}{true}}_{0}^{false(k, m + 1 false)}, …, {\overset{β}{true}}_{j}^{false(k, m + 1 false)}, {\overset{β}{true}}_{j + 1}^{false(k, m false)}, …, {\overset{β}{true}}_{d}^{false(k, m false)} false)}^{'} \in {double-struckR}^{1 + d} false(j = 0, 1, …, d false),

which represents the value of the m th “inner” iteration

{\overset{italicβ}{true}}^{false(k, m false)}

, at the time of the j th coordinate's update.…”

Section: Methodsmentioning

confidence: 99%

“…(In Equation ,

G_{i j}^{false(k false)} \in double-struckR

represents the j th element of

G_{i}^{false(k false)} \in {double-struckR}^{d + 1}

.) For Equation , the MMCD approach of Jiang and Huang () assumes a majorization of the “scaling” factor

false{\sum_{i = 1}^{n} w_{i}^{false(k false)} {false(G_{i j}^{false(k false)} false)}^{2} false}

attached to β j , by some constant M >0, that is,

\sum_{i = 1}^{n} w_{i}^{false(k false)} {false(G_{i j}^{false(k false)} false)}^{2} \leq M

, for all j =0,…, d (for each k ). (That this majorization is useful for solving Equation will be explained at the end of this section.)…”

Section: Methodsmentioning

confidence: 99%

“…For the intercept, that is, for j =0,

{\overset{β}{true}}_{0}^{false(k, m + 1 false)} = τ_{0} false/ M,

where

τ_{0} = M {\overset{β}{true}}_{0}^{false(k, m false)} + n^{- 1} \sum_{i = 1}^{n} G_{i 1}^{false(k false)} false(Y_{i} - F false(G_{i}^{{false(k false)}^{'}} {\overset{italicβ}{true}}^{false(k, m false)} false) false)

, in which

{\overset{italicβ}{true}}^{false(k, m false)} = {false({\overset{β}{true}}_{0}^{false(k, m false)}, {\overset{β}{true}}_{1}^{false(k, m false)}, …, {\overset{β}{true}}_{d}^{false(k, m false)} false)}^{'}

. The convergence of iteration (over the “inner” loop m ) defined by the rules and to a solution of the ( k th) equations of Equation (for each k ) is given by the Theorem 1 of Jiang and Huang (). (In Equation , (·) + denotes the thresholding operator.…”

Section: Methodsmentioning

confidence: 99%

“…We use the notationβ ðk;mÞ to keep track of the (mth) "inner" CD iteration, defined within each (kth) step of Equation (16). Jiang and Huang (2014) developed an efficient implementation of CD, termed the majorization minimization by CD (MMCD), which we employ in this paper to obtain Equation (16)…”

Section: Penalized Estimation Via Coordinate Descentmentioning

confidence: 99%

See 3 more Smart Citations

Logistic regression error‐in‐covariate models for longitudinal high‐dimensional covariates

Park

Lee

2019

Stat

View full text Add to dashboard Cite

We consider a logistic regression model for a binary response where part of its covariates are subject‐specific random intercepts and slopes from a large number of longitudinal covariates. These random effect covariates must be estimated from the observed data, and therefore, the model essentially involves errors in covariates. Because of high dimension and high correlation of the random effects, we employ longitudinal principal component analysis to reduce the total number of random effects to some manageable number of random effects. To deal with errors in covariates, we extend the conditional‐score equation approach to this moderate dimensional logistic regression model with random effect covariates. To reliably solve the conditional‐score equations in moderate/high dimension, we apply a majorization on the first derivative of the conditional‐score functions and a penalized estimation by the smoothly clipped absolute deviation. The method was evaluated through a set of simulation studies and applied to a data set with longitudinal cortical thickness of 68 regions of interest to identify biomarkers that are related to dementia transition.

show abstract

“…We use the notation

{\overset{italicβ}{true}}^{false(k, m false)}

{\overset{italicβ}{true}}_{j}^{false(k, m false)} = {false({\overset{β}{true}}_{0}^{false(k, m + 1 false)}, …, {\overset{β}{true}}_{j}^{false(k, m + 1 false)}, {\overset{β}{true}}_{j + 1}^{false(k, m false)}, …, {\overset{β}{true}}_{d}^{false(k, m false)} false)}^{'} \in {double-struckR}^{1 + d} false(j = 0, 1, …, d false),

which represents the value of the m th “inner” iteration

{\overset{italicβ}{true}}^{false(k, m false)}

, at the time of the j th coordinate's update.…”

Section: Methodsmentioning

confidence: 99%

“…(In Equation ,

G_{i j}^{false(k false)} \in double-struckR

represents the j th element of

G_{i}^{false(k false)} \in {double-struckR}^{d + 1}

.) For Equation , the MMCD approach of Jiang and Huang () assumes a majorization of the “scaling” factor

false{\sum_{i = 1}^{n} w_{i}^{false(k false)} {false(G_{i j}^{false(k false)} false)}^{2} false}

attached to β j , by some constant M >0, that is,

\sum_{i = 1}^{n} w_{i}^{false(k false)} {false(G_{i j}^{false(k false)} false)}^{2} \leq M

, for all j =0,…, d (for each k ). (That this majorization is useful for solving Equation will be explained at the end of this section.)…”

Section: Methodsmentioning

confidence: 99%

“…For the intercept, that is, for j =0,

{\overset{β}{true}}_{0}^{false(k, m + 1 false)} = τ_{0} false/ M,

where

τ_{0} = M {\overset{β}{true}}_{0}^{false(k, m false)} + n^{- 1} \sum_{i = 1}^{n} G_{i 1}^{false(k false)} false(Y_{i} - F false(G_{i}^{{false(k false)}^{'}} {\overset{italicβ}{true}}^{false(k, m false)} false) false)

, in which

{\overset{italicβ}{true}}^{false(k, m false)} = {false({\overset{β}{true}}_{0}^{false(k, m false)}, {\overset{β}{true}}_{1}^{false(k, m false)}, …, {\overset{β}{true}}_{d}^{false(k, m false)} false)}^{'}

Section: Methodsmentioning

confidence: 99%

Section: Penalized Estimation Via Coordinate Descentmentioning

confidence: 99%

See 2 more Smart Citations

Logistic regression error‐in‐covariate models for longitudinal high‐dimensional covariates

Park

Lee

2019

Stat

View full text Add to dashboard Cite

show abstract

“…Relaxing penalization of large entries of β ameliorates LASSO’s shrinkage. If one majorizes the MCP function q ( β i ) by a scaled absolute value function, then cyclic coordinate descent parameter updates resemble the corresponding LASSO updates (Jiang & Huang, 2011). …”

Section: Methodsmentioning

confidence: 99%

Iterative hard thresholding for model selection in genome‐wide association studies

Keys

Chen

Lange

2017

Genetic Epidemiology

View full text Add to dashboard Cite

A genome-wide association study (GWAS) correlates marker and trait variation in a study sample. Each subject is genotyped at a multitude of SNPs (single nucleotide polymorphisms) spanning the genome. Here we assume that subjects are randomly collected unrelateds and that trait values are normally distributed or can be transformed to normality. Over the past decade, geneticists have been remarkably successful in applying GWAS analysis to hundreds of traits. The massive amount of data produced in these studies presents unique computational challenges. Penalized regression with LASSO or MCP penalties is capable of selecting a handful of associated SNPs from millions of potential SNPs. Unfortunately, model selection can be corrupted by false positives and false negatives, obscuring the genetic underpinning of a trait. Here we compare LASSO and MCP penalized regression to iterative hard thresholding (IHT). On GWAS regression data, IHT is better at model selection and comparable in speed to both methods of penalized regression. This conclusion holds for both simulated and real GWAS data. IHT fosters parallelization and scales well in problems with large numbers of causal markers. Our parallel implementation of IHT accommodates SNP genotype compression and exploits multiple CPU cores and graphics processing units (GPUs). This allows statistical geneticists to leverage commodity desktop computers in GWAS analysis and to avoid supercomputing.

show abstract

Simultaneous Multiple Change Points Estimation in Generalized Linear Models

Sun

2020

Contemporary Experimental Design, Multivariate Analysis and Data Mining

View full text Add to dashboard Cite

Majorization minimization by coordinate descent for concave penalized generalized linear models

Cited by 12 publications

References 24 publications

Logistic regression error‐in‐covariate models for longitudinal high‐dimensional covariates

Logistic regression error‐in‐covariate models for longitudinal high‐dimensional covariates

Iterative hard thresholding for model selection in genome‐wide association studies

Simultaneous Multiple Change Points Estimation in Generalized Linear Models

Contact Info

Product

Resources

About