What Would It Take to Change an Inference? Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences

Frank, Kenneth A.; Maroulis, Spiro; Duong, Minh Q.; Kelcey, Benjamin

doi:10.3102/0162373713493129

Cited by 265 publications

(245 citation statements)

References 81 publications

Supporting

Mentioning

223

Contrasting

Unclassified

Order By: Relevance

“…In response, we quantify how much of the estimated effect of potential provider entropy must be due to sampling bias to invalidate our inference (Frank & Min, 2007). In particular, to invalidate our inference, one would have to replace one-third (about 7) of our schools with other schools in which there was no effect (Frank et al, 2013). 14 Note that our sample differed from state averages on percent free and reduced lunches and percent white by only 12 percent.…”

Section: Concerns About Sampling Bias (External Validity)mentioning

confidence: 99%

What Is a “Good” Social Network for Policy Implementation? The Flow of Know-How for Organizational Change

Frank¹,

Penuel²,

Krause

2015

J. Pol. Anal. Manage.

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Concerns About Sampling Bias (External Validity)mentioning

confidence: 99%

What Is a “Good” Social Network for Policy Implementation? The Flow of Know-How for Organizational Change

Frank¹,

Penuel²,

Krause

2015

J. Pol. Anal. Manage.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the estimates π 01j and π 11j , we tested the sensitivity of our inferences to plausible departures from the unconfoundedness assumption, using the procedure of Frank et al (2013). The goal of the sensitivity analysis is to account for the possibility of conditions that could invalidate our inferences due to bias from non-random assignment to treatment conditions, such as heterogeneous school policies or unmeasured parental decision factors.…”

Section: Sensitivity Analysesmentioning

confidence: 99%

“…Specifically, we determined the proportion of our estimate that would need to arise from bias in order to invalidate our inference, that is, the invalidating proportions (IP). Frank et al (2013) provide the following quantification of this proportion:…”

Section: Sensitivity Analysesmentioning

confidence: 99%

“…For assessing the bias of non-random treatment assignment on an estimated coefficient, Frank et al (2013) demonstrated that this proportion can be computed for an estimated effectd asd À ððSEðdÞx; t critical;df Þ=d) (the derivation as it pertains to non-random treatment assignment is detailed in Equations (11-15) in Frank et al (2013)). We applied this approach to our estimated average treatment effects using a critical value of 1.96 or -1.96 depending on the direction of the estimate, obtaining IPs ranging from 17.47 to 90.07%.…”

Section: Sensitivity Analysesmentioning

confidence: 99%

See 1 more Smart Citation

Effects of kindergarten retention for at-risk children’s mathematics development

Vandecandelaere

Schmitt

Vanlaar

et al. 2014

Research Papers in Education

View full text Add to dashboard Cite

When a child does not seem to be ready for primary school, a popular practice is to grant the child more time by letting it repeat kindergarten. However, previous quasi-experimental research demonstrated negative, though diminishing, effects of kindergarten retention on academic learning during the first years of primary school. The present study extends the existing evidence by addressing children's post-treatment school trajectories. Analysing data from a large-scale longitudinal study, we find that, on average, kindergarten repeaters would perform better for mathematics until five years later, were they promoted to first grade instead. However, if promoted instead, kindergarten repeaters would also have a higher likelihood to be retained in first grade and, under that condition, have a lower growth rate and score lower for mathematics five years later.

show abstract

“…Of course, no single research design is definitive. Most randomized controlled trials have great strength in establishing causality within the sample but caution is required in generalizing beyond the sample (15). In Asensio and Delmas's study, the sample is limited to university apartments in southern California, so replications are needed to see how altruistic appeals will work in other contexts.…”

Section: Next Stepsmentioning

confidence: 99%

Altruism, self-interest, and energy consumption

Dietz

2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

How can we encourage greater energy efficiency? Estimates that take account of behavioral plasticity-the ease with which actions can be taken-indicate that the United States could reduce overall greenhouse gas emissions by 7% if households adopted simple and money-saving efficiency actions (1). However, most households are not taking these actions despite the benefits for the environment and for household budgets. By engaging key factors that influence household energy consumption, Asensio and Delmas shed light on unrealized energy efficiency (2). Their results provide useful insights for the design of programs to encourage energy efficiency; in doing so, they hone our understanding of environmental decision making.The analysis of Asensio and Delmas uses real-time energy consumption data at the appliance level for 118 apartments in southern California over 9 mo. During the study, the authors provided two kinds of feedback to encourage efficiency. Some randomly assigned households were given weekly feedback on their annual dollar cost-savings compared with the most efficient 10% of their neighbors. The other households received weekly feedback about their consumption in the metric of reduced air pollution emissions rather than as dollar costs to the household. That is, one group received information about how efficiency was serving their self-interest and the other about how efficiency was contributing to the common good of reduced air pollution.The results are intriguing. Households who received the air pollution message reduced their electrical consumption by 8.2% over the 100-d experimental monitoring period. Those in this group who had children changed their behavior even more dramatically, achieving a 19% reduction in consumption. Information about air pollution reductions reduced consumption in all but the 10% of households with the smallest energy use. In contrast, the households that received the monetary messages actually increased their consumption, and that increase was greater for households with children and for those in the highest use categories. What Motivates Consumption?Many programs intended to encourage energy efficiency assume that monetary costs and the amenities provided by energy are the dominant motivators of energy consumption. Asensio and Delmas's results fly in the face of this logic. However, the findings are consistent with our broader understanding of environmental decision making and environmentally significant consumption (3-5).By engaging key factors that influence household energy consumption, Asensio and Delmas shed light on unrealized energy efficiency.One motivation for household energy use is obvious. We use energy to enhance wellbeing. Space heating and cooling, food refrigeration, cooking heat, hot water, lighting, and access to a vast array of information technologies all enhance well-being. However, for most households' energy uses there are diminishing returns to well-being with increased consumption. We don't want our rooms too hot or too cold, nor do we benefit from t...

show abstract

What Would It Take to Change an Inference? Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences

Cited by 265 publications

References 81 publications

What Is a “Good” Social Network for Policy Implementation? The Flow of Know-How for Organizational Change

What Is a “Good” Social Network for Policy Implementation? The Flow of Know-How for Organizational Change

Effects of kindergarten retention for at-risk children’s mathematics development

Altruism, self-interest, and energy consumption

Contact Info

Product

Resources

About