Proceeding From Observed Correlation to Causal Inference: The Use of Natural Experiments

Abstract

ABSTRACT-This article notes five reasons why a correlation between a risk (or protective) factor and some specified outcome might not reflect environmental causation. In keeping with numerous other writers, it is noted that a causal effect is usually composed of a constellation of components acting in concert. The study of causation, therefore, will necessarily be informative on only one or more subsets of such components. There is no such thing as a single basic necessary and sufficient cause. Attention is drawn to the need (albeit unobservable) to consider the counterfactual (i.e., what would have happened if the individual had not had the supposed risk experience). Fifteen possible types of natural experiments that may be used to test causal inferences with respect to naturally occurring prior causes (rather than planned interventions) are described. These comprise five types of genetically sensitive designs intended to control for possible genetic mediation (as well as dealing with other issues), six uses of twin or adoptee strategies to deal with other issues such as selection bias or the contrasts between different environmental risks, two designs to deal with selection bias, regression discontinuity designs to take into account unmeasured confounders, and the study of contextual effects. It is concluded that, taken in conjunction, natural experiments can be very helpful in both strengthening and weakening causal inferences.From an early point in their training, all behavioral scientists are taught that statistically significant correlations do not necessarily mean any kind of causative effect. Nevertheless, the literature is full of studies with findings that are exclusively based on correlational evidence. Researchers tend to fall into one of two camps with respect to how they react to the problem. First, there are those who are careful to use language that avoids any direct claim for causation, and yet, in the discussion section of their papers, they imply that the findings do indeed mean causation. Second, there are those that completely accept the inability to make a causal inference on the basis of simple correlation or association and, instead, take refuge in the claim that they are studying only associations and not causation. This second, ''pure'' approach sounds safer, but it is disingenuous because it is difficult to see why anyone would be interested in statistical associations or correlations if the findings were not in some way relevant to an understanding of causative mechanisms.Some researchers argue that only laboratory experiments or randomized controlled trials (RCTs) allow any firm causal inference. Of course, it is true that both provide a much needed control that allows rigorous testing of the causal hypothesis and takes account of unmeasured confounders. Nevertheless, there are many risk factors for which neither laboratory experiments nor RCTs are feasible or ethical. That would apply, for example, to experiences such as maltreatment, life stresses, or child neglect. How should thes...

The End of Behavioral Genetics?

“…The difficulty with this latter strategy of course is that experiments of nature depend on serendipitous events that are outside the investigator's control. Rutter (Rutter, 2007;Rutter, Pickles, Murray, & Eaves, 2001) has recently shown, however, that several behavioral genetic research designs constitute experiments of nature and consequently provide unique opportunities for rigorous evaluation of causal hypotheses with observational data. We illustrate this aspect of contemporary behavioral genetic research with two examples of recent MCTFR research.…”

Smoking in Pregnancy and Child ADHD

“…[4][5][6] Several approaches can be used to control for unmeasured confounding, and convergence of findings from different methods and cohorts provides the strongest support for causal claims. 7,8 In sibling control designs, siblings discordant for maternal smoking during pregnancy are compared with respect to ADHD. Some studies using this approach have suggested that the association between maternal smoking during pregnancy and offspring ADHD is due to unmeasured confounding, 9 -12 whereas a recent study suggested that the association may be causal.…”

Achievements and challenges in the biology of environmental effects

“…There is no point in studying the biological embedding of experiences whose risk effects are not environmentally mediated. It is all too often assumed that the remedy lies in the statistical control of confounders, but numerous studies using "natural experiment" designs have shown that reliance on such statistical control is a seriously flawed approach (4,5). Nevertheless, there is good evidence that physical and sexual abuse, as well as poverty, does have adverse causal effects.…”

“…First, there are about a dozen different natural experiments that have been shown to be useful (2,5,33). Each involves a set of assumptions and a mixture of strengths and limitations; however, taken together, they have substantial power.…”

“…Third, even though an experience can be appropriately viewed as describing an environment, as with, for example, family break-up or poverty, are the risk effects environmentally (rather than genetically) mediated (3)? That issue emphasizes the crucial importance of considering the research strategies that may be used to test whether it is justifiable to proceed from a statistical association or correlation to a causal inference (4,5).…”