Unobservable Selection and Coefficient Stability: Theory and Validation

Abstract

Inferring causal treatment effects in the presence of possible omitted variable bias is as well-known problem. Altonji, Elder and Taber (2005) suggest that the degree of selection on observable variables might be used as a guide to the remaining bias in controlled regressions. I expand on their setup and demonstrate how, with an equal selection assumption, a causal effect can be recovered using coefficients, R-squared values from controlled and uncontrolled regressions and an estimate of the iid noise in the outcome. I discuss the relationship between this technique and the heuristic procedure of adding sequential controls until coefficients stabilize. I consider two validation exercises which explore whether coefficients adjusted in this way are closer to the truth. First, I estimate the impact of early life and prenatal influences on child IQ. Simple controlled regressions give misleading estimates; the bias-adjusted estimates are significantly better. However, I show that the progressive adding of controls in this setting is not effective. Second, I match observational and randomized trial data for 31 treatment-outcome pairs in three public health settings. I estimate a best-fit degree of adjustment across all settings (the free parameter is the amount of iid noise in the outcome). I show that the bias-adjusted coefficients preform much better than simple controlled coefficients; the total error reduction is 30% and a number of false-positive results are rejected without significant loss in true-positive results. The magnitude of the best-fit adjustment suggests adjusting the controlled coefficient by approximately the same amount as the movement between uncontrolled and controlled coefficients.

The Wild West IS Wild: The Homicide Resource Curse

“…We have so far included all types of minerals in empirical 20 Note that 1.13 exceeds 1, the critical value recommended by Oster (2015). We follow the recommendation by Oster (2015) and assume that the maximum achievable R-squared exceeds by 30% the one obtained when including all observable covariates. With the most conservative hypothesis, i.e.…”

Is there an educational penalty for being suspended from school?

“…The results in Table 3 show that the estimated effects of suspension on all three education measures are highly sensitive to the degree of correlation imposed on the 17 Because two of our three models are non-linear, and because conclusions in the remaining linear model (for university entrance score) are highly sensitive to the value assumed for R max , we do not implement the related Oster (2013) approach to infer selection bias by examining coefficient stability and movements in R 2 as additional controls are added to the model. unobserved determinants of suspension and educational attainment or achievement.…”

Family Instability and Locus of Control in Adolescence

“…Since the data comprise only post-treatment non-cognitive skills of mothers, we also estimate the robustness of our results to omitted variables bias by applying a novel econometric method proposed by Oster (2013). This method allows assessing the bias resulting from unobservables.…”