A note on standardization in penalized regressions

Lee, Sangin

doi:10.7465/jkdi.2015.26.2.505

Cited by 6 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the LASSO or SCAD method may not be directly applicable to the high dimensional case with p * > n (Zou and Hastie, 2005;Lee et al, 2010;Fan and Lv, 2010;Lee, 2015). With the HL penalty method, extension to such high dimensional case in HGLMs would be an interesting topic.…”

Section: Discussionmentioning

confidence: 99%

Variable selection in Poisson HGLMs using h-likelihoood

Ha¹,

Cho²

2015

Journal of the Korean Data and Information Science Society

View full text Add to dashboard Cite

Selecting relevant variables for a statistical model is very important in regression analysis. Recently, variable selection methods using a penalized likelihood have been widely studied in various regression models. The main advantage of these methods is that they select important variables and estimate the regression coefficients of the covariates, simultaneously. In this paper, we propose a simple procedure based on a penalized h-likelihood (HL) for variable selection in Poisson hierarchical generalized linear models (HGLMs) for correlated count data. For this we consider three penalty functions (LASSO, SCAD and HL), and derive the corresponding variable-selection procedures. The proposed method is illustrated using a practical example.

show abstract

Section: Discussionmentioning

confidence: 99%

Variable selection in Poisson HGLMs using h-likelihoood

Ha¹,

Cho²

2015

Journal of the Korean Data and Information Science Society

View full text Add to dashboard Cite

show abstract

“…Self‐reported personality nuanced characteristics were obtained with the 44 items from the Big‐Five Inventory (BFI; John & Srivastava, 1999), answered on a 5‐point scale ranging from 1 ( strongly disagree ) to 5 ( strongly agree ). To minimize prediction differences between items that stem only from the difference in the items' variances, ratings were standardized so that each item's mean would be zero and SD would be 1 (Friedman et al, 2010; Lee, 2015).…”

Section: Methodsmentioning

confidence: 99%

The empathic personality profile: Using personality characteristics to reveal genetic, environmental, and developmental patterns of adolescents' empathy

Abramson

Eldar

Markovitch

et al. 2022

Journal of Personality

View full text Add to dashboard Cite

Objective How do genetic and environmental processes affect empathy during early adolescence? This study illuminated this question by examining the aetiology of empathy with the aetiology of other personality characteristics. Method Israeli twin adolescents rated their empathy and personality at ages 11 (N = 1176) and 13 (N = 821) (733 families, 51.4% females). Parents rated adolescents' emotional empathy. Adolescents performed an emotion recognition task, indicating cognitive empathy. Results Using a cross‐validated statistical learning algorithm, this study found emotional and cognitive “empathic personality profiles,” which describe and predict self‐reported empathy from nuanced Big‐Five personality characteristics, or “nuances” (i.e., individual items). These profiles predicted empathy moderately (R2 = 0.17–0.24) and were stable and robust, within each age and between ages. They also predicted empathy in a new sample of older nontwin adolescents (N = 96) and were validated against non‐self‐report empathy measures. Both emotional and cognitive empathy were predicted by nuances representing positive attitudes toward others, trust, forgiveness, and openness to experiences. Emotional empathy was also predicted by nuances representing anxiousness and negative reactivity. Twin analyses revealed overlapping genetic and environmental influences on empathy and the empathic personality profiles and overlapping environmental influences on empathy–personality change. Conclusions This study demonstrates how addressing the complexity of individuals' personalities can inform adolescents' empathy development.

show abstract

“…If variables are on different scales, this can lead to uneven penalization of coefficients in which variables with smaller standard deviations are biased more strongly towards zero. 33 If the scale of predictor x is increased by a factor 10, its regression coefficient is reduced by a factor 10, bringing it closer to zero where it will be more affected by penalization. Standardization is a widely used method for equalizing predictor scales.…”

Section: Standardizing Predictorsmentioning

confidence: 99%

“…As explained in Formula (), regularization penalizes all coefficients equally, without regard for their scale. If variables are on different scales, this can lead to uneven penalization of coefficients in which variables with smaller standard deviations are biased more strongly towards zero 33 . If the scale of predictor

x

is increased by a factor 10, its regression coefficient is reduced by a factor 10, bringing it closer to zero where it will be more affected by penalization.…”

Section: Introductionmentioning

confidence: 99%

Selecting relevant moderators with Bayesian regularized meta‐regression

Lissa

Erp

Clapper

2023

Research Synthesis Methods

View full text Add to dashboard Cite

When meta-analyzing heterogeneous bodies of literature, meta-regression can be used to account for potentially relevant between-studies differences. A key challenge is that the number of candidate moderators is often high relative to the number of studies. This introduces risks of overfitting, spurious results, and model non-convergence. To overcome these challenges, we introduce Bayesian Regularized Meta-Analysis (BRMA), which selects relevant moderators from a larger set of candidates by shrinking small regression coefficients towards zero with regularizing (LASSO or horseshoe) priors. This method is suitable when there are many potential moderators, but it is not known beforehand which of them are relevant. A simulation study compared BRMA against state-of-the-art random effects meta-regression using restricted maximum likelihood (RMA). Results indicated that BRMA outperformed RMA on three metrics: BRMA had superior predictive performance, which means that the results generalized better; BRMA was better at rejecting irrelevant moderators, and worse at detecting true effects of relevant moderators, while the overall proportion of Type I and Type II errors was equivalent to RMA. BRMA regression coefficients were slightly biased towards zero (by design), but its residual heterogeneity estimates were less biased than those of RMA. BRMA performed well with as few as 20 studies, suggesting its suitability as a small sample solution. We present free open source software implementations in the R-package pema (for penalized meta-analysis) and in the stand-alone statistical program JASP. An applied example demonstrates the use of the R-package.

show abstract

A note on standardization in penalized regressions

Cited by 6 publications

References 15 publications

Variable selection in Poisson HGLMs using h-likelihoood

Variable selection in Poisson HGLMs using h-likelihoood

The empathic personality profile: Using personality characteristics to reveal genetic, environmental, and developmental patterns of adolescents' empathy

Selecting relevant moderators with Bayesian regularized meta‐regression

Contact Info

Product

Resources

About