The fragility of standard inferential approaches in principal stratification models relative to direct likelihood approaches

Abstract

Many empirical settings involve the specification of models leading to complicated likelihood functions, for example, finite mixture models that arise in causal inference when using Principal Stratification (PS). Traditional asymptotic results cannot be trusted for the associated likelihood functions, whose logarithms are not close to being quadratic and may be multimodal even with large sample sizes. We first investigate the shape of the likelihood function with models based on PS by providing diagnostic tools for evaluating ellipsoidal approximations based on the second derivatives of the log‐likelihood at a mode. In these settings, inference based on standard approximations is inappropriate, and other forms of inference are required. We explore the use of a direct likelihood approach for parsimonious model selection and, specifically, propose comparing values of scaled maximized likelihood functions under competitive models to select preferred models. An extensive simulation study provides guidelines, for calibrating the use of scaled log‐likelihood ratio statistics, as functions of the complexity of the models being compared.

Principal Stratification Analysis Using Principal Scores

“…Therefore, they have superior statistical properties compared with principal stratification analysis based on normal mixture models, which have unbounded likelihood and inaccurate asymptotic normal distribution approximations as pointed out by Frumento et al . (). Furthermore, in our simulation studies which are shown in the on‐line supplementary material, we compare our method with the method of Jo and Stuart () involving outcome modelling and find that our estimator is not only robust to misspecification of the outcome model but also has smaller standard error.…”

“…The tremendous variability of the estimator is due to the unstable numerical issue and unreliable large sample normal distribution approximation, as investigated by Frumento et al . ().…”

“…Without an ER, Zhang et al (2009) and Frumento et al (2012) assumed normal distribution outcome models within principal strata, and thus identifiability of the causal effects within principal strata is ensured by identifiability of the normal mixture model. Unfortunately, the results are sensitive to the parametric modelling assumption, and the unbounded likelihood function jeopardizes statistical inference even with correctly specified models (Ding et al, 2011;Frumento et al, 2016). Without these assumptions, however, large sample bounds of causal effects are often too wide to be informative (Zhang and Rubin, 2003;Cheng and Small, 2006).…”

“…However, applying the normal mixture model of Zhang et al (2009) we obtain point estimate 12:34 with standard error 47:17. The tremendous variability of the estimator is due to the unstable numerical issue and unreliable large sample normal distribution approximation, as investigated by Frumento et al (2016).…”

Compared to what? Variation in the impacts of early childhood education by alternative care type

“…Incorporating this structure is immediate with a Bayesian approach but can prove quite complex in a likelihood setting. In addition, accounting for uncertainty in the parameter estimates is natural with a Bayesian approach but can be more involved with a direct likelihood approach (see, for example, Frumento et al, 2016). While we use a Bayesian estimation approach, we would expect quite similar results using either method.…”