Fixed Effects and Post-Treatment Bias in Legacy Studies

Abstract

A growing literature examines how historical institutions influence contemporary political attitudes and behavior. Recent work has argued that these studies need to properly account for spatial heterogeneity by incorporating regional fixed effects. Here, we discuss the theoretical and empirical obstacles that have to be addressed to properly incorporate fixed effects in legacy studies. We illustrate our arguments using Pepinsky et al.'s (2020) reassessment of a recent study on the long-term effects of concentration camps in Germany (Homola et al. 2020). We show that Pepinsky et al. incorporate fixed effects incorrectly and, as a result, their analysis suffers from post-treatment bias. We further demonstrate that results from the original article remain substantively the same when we incorporate regional fixed effects correctly. Finally, simulations reveal that camp proximity consistently outperforms spatially correlated noise in this specific study.

Fixed Effects and Post-Treatment Bias in Legacy Studies

“…To overcome this issue, PGZ add contemporary state fixed effects to our models and find that proximity to concentration camps is no longer a reliable predictor of intolerance. Previously we noticed that the authors' strategy risked inducing post-treatment bias and that accounting for geographical heterogeneity with Weimar-era state fixed effects (the state borders in place when the camps were created) avoided this problem and reproduced the findings described in HPT (Homola et al 2020b). PGZ now return to this issue and suggest that contemporary states do not bias the estimates if the following assumptions hold: (a) contemporary cross-state heterogeneity is not in the causal path between camp proximity and contemporary attitudes, and (b) there are no unobserved variables that jointly explain contemporary state differences and contemporary outgroup intolerance or camp proximity.…”

Modeling Spatial Heterogeneity and Historical Persistence: Nazi Concentration Camps and Contemporary Intolerance

“…Accordingly, they control for urbanism, being in the former West Germany, and other confounders when estimating the effect of camp proximity on attitudes. Yet, HPT do not account for potential differences in attitudes across Germany's federal states ([Bundes-] Länder), dismissing Germany's regional heterogeneity as a posttreatment variable (Homola, Pereira, and Tavits 2021). Clarifying the conceptual foundations of posttreatment bias and reviewing the historical record on postwar state creation, we argue that state-level differences confound the relationship between distance to camps and out-group intolerance.…”

“…But HPT do not explain their choice to ignore statelevel heterogeneity, arguing elsewhere that controlling for contemporary administrative boundaries produces posttreatment bias (Homola, Pereira, and Tavits 2021). Indeed, there are two conditions when one should not control for variables that are correlated with both treatment and outcome: 2 (1) if a confounder lies along the causal path from treatment to outcome, then controlling for it will generate posttreatment bias that masks the causal relationship of interest and (2) controlling for a variable that is a causal consequence of both the determinants of the causal variable and the outcome generates "M-bias," a form of collider bias.…”

Pandemics and Political Development

“…4. 106 This strategy is based on observations that the Black Death was most severe in the spring and summer and that its intensity waned over time;Aberth 2021, p. 26;Benedictow 2004;Gottfried 1983;Campbell 2016.107 Ahmed and Stasavage 2020.108 Regarding the use of spatial fixed effects in legacy studies, see the contributions to this debate byPepinsky, Goodman, and Ziller 2020;Homola, Pereira, and Tavits 2020a;Homola, Pereira, and Tavits 2020b.…”

“…108Regarding the use of spatial fixed effects in legacy studies, see the contributions to this debate by Pepinsky, Goodman, and Ziller 2020; Homola, Pereira, and Tavits 2020a; Homola, Pereira, and Tavits 2020b.…”