Identifying Baseline Covariates for Use in Propensity Scores: A Novel Approach Illustrated for a Nonrandomized Study of Recovery High Schools

Abstract

There are many situations where random assignment of participants to treatment and comparison conditions may be unethical or impractical. This article provides an overview of propensity score techniques that can be used for estimating treatment effects in non-randomized quasi-experimental studies. After reviewing the logic of propensity score methods, we call attention to the importance of the strong ignorability assumption and its implications. We then discuss the importance of identifying and measuring a sufficient set of baseline covariates upon which to base the propensity scores and illustrate approaches to that task in the design of a study of recovery high schools for adolescents treated for substance abuse. One novel approach for identifying important covariates that we suggest and demonstrate is to draw on the predictor-outcome correlations compiled in meta-analyses of prospective longitudinal correlations.

Recovery high schools: Effect of schools supporting recovery from substance use disorders

“…Structured Clinical Interview for major Axis I psychiatric disorder diagnostics (26)); alcohol use, tobacco use, other substance use, and mental health treatment service receipt (all collected using the Timeline Followback method (25)); physical health treatment receipt, life satisfaction, sources of life stress, criminal justice system involvement, and school problems (all collected using the GAIN-Q3 (27)); perceived consequences of drug use (collected using the Personal Experiences Inventory (28)), interest in attending an RHS, perceived teacher support for substance use recovery, prior year school attendance and grades, negative attitudes toward school (collected using the Behavior Assessment System for Children (29)), perceived academic abilities, family income, parental education level, parental history of substance use treatment, and family history of mental health problems. Additional measurement details and rationale for selection of control variables has been reported in prior articles (24, 30).…”

“…Because adolescents self-selected into the intervention and comparison conditions, we used propensity scores to address potential baseline non-equivalence between the two conditions (30). Propensity scores were conceptualized as the predicted probability of attending an RHS, estimated with multilevel logistic regression models that included baseline values of all outcome variables as well as the covariate controls noted above.…”

Parents as interventionists: Addressing adolescent substance use

“…The program's length (3 sessions, with one session conducted per week) was guided by the intention to stay within the parameters of a typical brief intervention (see Tanner-Smith & Lipsey, 2014), and yet to be comprehensive enough to address a range of problems. As noted previously, Bis in the literature and in practice can be as brief as a single 5-minute conversation, whereas some consist of 4 to 5 hour-long sessions.…”

Recovery High Schools: Students and Responsive Academic and Therapeutic Services

“…Dr. Finch has also published a manual for individuals planning to start a RHS (Finch, 2005). We are currently conducting a rigorous research study that examines academic and therapeutic services and outcomes of RHS programs, comparing students entering RHS programs after treatment to students who enrolled in other educational settings (Tanner-Smith & Lipsey, 2014). The present article draws from these prior studies, focusing on how academic and therapeutic programs are structured to meet the needs of the students in RHSs.…”