Treatment effect validation via a permutation test in Stata

Abstract

In this paper we describe the testing procedure for assessing the statistical significance of treatment effect under the experimental conditions of baseline imbalance across covariates and attrition from the survey, using the permutation tests proposed by Freedman and Lane (1983) and Romano and Wolf (2016). We discuss the testing procedure for these hypotheses based on a linear regression model and introduce the new Stata command [R] permtest for the implementation of the permutation test in Stata. Moreover, we investigate the finite-sample performance as well as the statistical validity of the test with a Monte Carlo simulation study in which we examine the empirical size and power properties under the conditions of baseline imbalance and attrition for a fixed number of permutation steps.