Sparse modeling of categorial explanatory variables

Gertheiss, Jan; Tutz, Gerhard

doi:10.1214/10-aoas355

Cited by 83 publications

(97 citation statements)

References 15 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For large most of the parametersˇr will be estimated as being very similar. Fusion penalty terms of this form have been considered, for example, by Bondell and Reich (2009), Gertheiss and Tutz (2010), and Tutz and Oelker (2015).…”

Section: Fixed-effects Modelmentioning

confidence: 99%

Modeling Discrete Time-to-Event Data

Tutz¹,

Schmid²

2016

Springer Series in Statistics

Self Cite

125

180

View full text Add to dashboard Cite

Section: Fixed-effects Modelmentioning

confidence: 99%

Modeling Discrete Time-to-Event Data

Tutz¹,

Schmid²

2016

Springer Series in Statistics

Self Cite

125

180

View full text Add to dashboard Cite

“…() for ordered predictors. The use for categorical predictors has been propagated by Bondell and Reich () for factorial designs and as a selection tool by Gertheiss and Tutz ().…”

Section: Regularised Estimation For Group‐specific Modelsmentioning

confidence: 99%

Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures

Tutz

Oelker

2016

Int Statistical Rev

Self Cite

View full text Add to dashboard Cite

Although each statistical unit on which measurements are taken is unique, typically there is not enough information available to account totally for its uniqueness. Therefore, heterogeneity among units has to be limited by structural assumptions. One classical approach is to use random effects models, which assume that heterogeneity can be described by distributional assumptions. However, inference may depend on the assumed mixing distribution, and it is assumed that the random effects and the observed covariates are independent. An alternative considered here is fixed effect models, which let each unit has its own parameter. They are quite flexible but suffer from the large number of parameters. The structural assumption made here is that there are clusters of units that share the same effects. It is shown how clusters can be identified by tailored regularised estimators. Moreover, it is shown that the regularised estimates compete well with estimates for the random effects model, even if the latter is the data generating model. They dominate if clusters are present.

show abstract

“…0 1 2 3 4 5 6 7 8 9 V7 0 1 2 3 4 5 V8 0 1 2 V9 0 1 2 3 4 5 6 7 8 9 10 11 12 V10 0 1 2 3 4 8 11 12 17 5 6 7 9 10 13 14 15 16 18 19 20 21 22 23 24 For covariates V7 and V8, the same levels as in the model selected in Gertheiss and Tutz (2010), which we will denote by Model (2) are fused. However, fusion is more pronounced in the Bayesian version for the ordinal covariates V6 (7 vs. 8 groups) and V9 (6 vs. 7 groups), and particularly for the nominal covariate V10, with only two groups under Bayesian fusion and 10 groups using regularization, see also Table 2.…”

Section: V6mentioning

confidence: 99%

Discussion: Bayesian regularization and effect smoothing for categorical predictors

Wagner

Pauger

2016

Statistical Modelling

View full text Add to dashboard Cite

IntroductionWe would first like to thank Gerhard Tutz and Jan Gertheiss for their profound review of regularization methods for categorical variables, either as covariates or as response variables in regression models. Categorical variables are rather the rule than the exception in regression analyses, particularly in the medical, social and economic sciences. It is amazing that it took quite a long time from ridge regression (Hoerl and Kennard, 1970) to versions of regularization methods, which take into account the specific structure of categorical covariates. As Tutz and Gertheiss predominantly discuss penalized estimation, we will focus on the Bayesian perspective to regularization and effect fusion for categorical covariates.Regularization and sparsity in regression type models have been addressed from a Bayesian point of view in the literature, see for example, Fahrmeir et al. (2010) for an overview on Bayesian regularization and George and McCulloch (1995), Ishwaran and Rao (2005) and O'Hara and Sillanpäa (2009) for approaches to Bayesian variable selection. However, the specific issues arising for categorical covariates have not yet received much attention, with the exception of Chipman (1996), who considers a Bayesian approach for groupwise selection of level effects of a categorical covariate (the Bayesian pendant to Section 3.2.1 of Tutz and Gertheiss, 2016) and the recently proposed Bayesian pendant to smoothing predictors and effect fusion based on spike and slab priors in Pauger and Wagner (2016).Here, we will discuss the relation between regularization penalties and prior distributions and then describe a Bayesian approach for smoothing of level effects of categorical covariates. Finally, we will briefly describe spike and slab prior distributions which encourage a sparse representation of their effects.

show abstract

Sparse modeling of categorial explanatory variables

Cited by 83 publications

References 15 publications

Modeling Discrete Time-to-Event Data

Modeling Discrete Time-to-Event Data

Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures

Discussion: Bayesian regularization and effect smoothing for categorical predictors

Contact Info

Product

Resources

About