Sample Size Calculation in Economic Experiments

Abstract

Clinical studies and economic experiments are often conducted with randomized controlled trials. In clinical studies, power calculations are carried out as a standard. But what’s about economic experiments? After describing the basic idea of the calculation procedure in a brief tutorial, I tackle the practice of sample size calculations in the field of experimental economics by considering the publications of 5 economic journals in the period 2000–2018. These are two top-ranked economic journals (Quarterly Journal of Economics and American Economic Review), the leading field journals in the area of experimental economics (Experimental Economics) and behavioral sciences (Journal of Economic Behavior and Organization), and a leading field journal in environmental economics (Environmental and Resource Economics). In contrast to clinical drug trials, sample size calculations have rarely been carried out by experimental economists. But the number of power calculations has slightly increased in recent years, especially in the top-ranked journals of economics. However, this can be partly explained by the fact that field experiments (in which scholars pay more attention to power analyses than in lab experiments these days) play an important role in these journals.

Does the mainland China–Hong Kong exchange program change the views of local university students in Hong Kong on regional cooperation? A randomized control-group pre-test post-test experiment

“…After excluding students who were either postgraduate students, part-time students, or non-local students, 44 and 37 students were enrolled in the intervention and control groups, respectively. As the calculation of the required sample size depends on the outcome variable of this study (students’ views on regional cooperation) ( Dong and Maynard, 2013 ; Gruener, 2020 ), and the outcome variable has not been quantified before, we have to rely on expert estimates to determine the required sample size. It is suggested that a sample of 10–20 can yield relatively high statistical power to detect moderate differences between groups for experimental research in the literature ( Roscoe, 1975 ; Sekaran and Bougie, 2016 ).…”