Individualized Multidirectional Variable Selection

Abstract

In this paper we propose a heterogeneous modeling framework which achieves individualwise feature selection and individualized covariates' effects subgrouping simultaneously. In contrast to conventional model selection approaches, the new approach constructs a separation penalty with multi-directional shrinkages, which facilitates individualized modeling to distinguish strong signals from noisy ones and selects different relevant variables for different individuals. Meanwhile, the proposed model identifies subgroups among which individuals share similar covariates' effects, and thus improves individualized estimation efficiency and feature selection accuracy. Moreover, the proposed model also incorporates within-individual correlation for longitudinal data to gain extra efficiency. We provide a general theoretical foundation under a double-divergence modeling framework where the number of individuals and the number of individual-wise measurements can both diverge, which enables inference on both an individual level and a population level. In particular, we establish strong oracle property for the individualized estimator to ensure its optimal large sample property under various conditions. An efficient ADMM algorithm is developed for computational scalability. Simulation studies and applications to post-trauma mental disorder analysis with genetic variation and an HIV longitudinal treatment study are illustrated to compare the new approach to existing methods.

Query-augmented Active Metric Learning

“…where h T i is the ith row of H, and h ik ∈ [0, 1] represents the probability that the ith sample belongs to the kth cluster. The penalty term min (|h ik |, |h ik − 1|) is a multi-directional separation penalty (MDSP) [32], which penalizes h ij to either 0 or 1 depending on the magnitude of h ij . The purpose of adding the MDSP penalty is to prevent strong signals from being pulled towards zero in the process of shrinking weak signals for sparsity pursuit, and thus to reduce the uncertainty on the cluster membership of each instance.…”

Heterogeneous Variable Selection in Nonlinear Panel Data Models: A Semi-Parametric Bayesian Approach

“…Wu et al (2012) and Zhao et al (2015), where the used term is often heterogeneous feature selection or sparsification. In contrast to the approach developed in Tang et al (2020), these latter approaches shrink the corresponding unit-specific parameter to zero in case a variable is selected, and not to some underlying population distribution shared across units.…”

“…A number of papers have proposed approaches to accommodate heterogeneous variable selection. They have done so for multivariate linear models (Kim et al, 2009, Tang et al, 2020, multivariate binary probit models (Kim et al, 2018), and multinomial logit models (Gilbride et al, 2006, Scarpa et al, 2009, Hensher and Greene, 2010Hole, 2011, Campbell et al, 2011, Hess et al, 2013, Hole et al, 2013. Few of these papers use a Bayesian approach (Gilbride et al, 2006, Kim et al, 2009, Kim et al, 2018.…”

“…Already when there are more than four variables, these approaches can run into problems. 2 To avoid overfitting, Tang et al (2020) use a penalization framework. They propose a linear model where each unit-specific parameter comes from a univariate discrete distribution with multiple possible outcomes of which one outcome is set to zero.…”

Subgroup identification via homogeneity pursuit for dense longitudinal/spatial data

“…The collection of dense longitudinal data makes it feasible to consider each patient to be a subgroup rather than only a member of the population. In other words, individualized modeling is possible since the number of repeated measurements is abundant. As a result, the coefficients of covariates in regression models may be allowed to be different for each of the patients.…”