Penalized Regression with Ordinal Predictors

Abstract

Ordered categorial predictors are a common case in regression modeling. In contrast to the case of ordinal response variables, ordinal predictors have been largely neglected in the literature. In this article penalized regression techniques are proposed. Based on dummy coding two types of penalization are explicitly developed; the first imposes a difference penalty, the second is a ridge type refitting procedure. A Bayesian motivation as well as alternative ways of derivation are provided. Simulation studies and real world data serve for illustration and to compare the approach to methods often seen in practice, namely linear regression on the group labels and pure dummy coding. The proposed regression techniques turn out to be highly competitive. On the basis of GLMs the concept is generalized to the case of non-normal outcomes by performing penalized likelihood estimation.

Statistical validation of the brief International Classification of Functioning, Disability and Health Core Set for osteoarthritis based on a large international sample of patients with osteoarthritis

“…This study confirms that the modification of group lasso by Gertheiss et al is useful when analyzing ICF data collected with an ordinal scale, such as the qualifier scale (21,22). The analyses of ICF data often have been presented as a challenge in the literature (35).…”

“…To identify the set of ICF categories that best explain patients' functioning, group lasso regression modified by Gertheiss and colleagues was applied using GH as the dependent variable and all ICF categories contained in the Comprehensive ICF Core Set for OA and some sociodemographic and disease-specific characteristics as independent variables (21,22). Group lasso is a regression technique that, in addition to the estimation of regression coefficients, allows for the selection of dummycoded categorical independent variables that best explain the variance of a dependent variable (37).…”

“…However, since the ICF Core Set is to be used as an international standard, additional studies including persons from different countries are needed. Moreover, recently, regression methods that allow simultaneously analyzing a large number of independent ordinal variables have been developed (21,22). This development presents the opportunity of validating the content of the Brief ICF Core Set for OA with a statistical method tailored to the data collected with ICF categories and using the ICF qualifier scale.…”

Variable selection for discrete competing risks models

“…The first term in (7) uses that time intervals are ordered. Therefore, for each cause r, differences between coefficients of adjacent time periods are penalized in a similar way as in penalized splines (Eilers and Marx, 1996) and regression with ordered predictors (Gertheiss and Tutz, 2009). The penalty controls how quickly hazard rates can change and hence smooths them over time.…”

Multinomial logit models with implicit variable selection

“…By penalizing such differences, one gets a smoother coefficient vector and avoid the high jumps among the parameter estimates corresponding to the ordinal covariate (see Gertheiss and Tutz (2009)). Let again the ordinal predictor take K + 1 categories 1, .…”