Estimation of the Conditional Variance in Paired Experiments

Abstract

In paired randomized experiments units are grouped in pairs, often based on covariate information, with random assignment within the pairs. Average treatment effects are then estimated by averaging the within-pair differences in outcomes. Typically the variance of the average treatment effect estimator is estimated using the sample variance of the within-pair differences. However, conditional on the covariates the variance of the average treatment effect estimator may be substantially smaller. Here we propose a simple way of estimating the conditional variance of the average treatment effect estimator by forming pairs of-pairs with similar covariate values and estimating the variances within these pairs-of-pairs. Even though these within-pairs-of-pairs variance estimators are not consistent, their average is consistent for the conditional variance of the average treatment effect estimator and leads to asymptotically valid confidence intervals.

Adaptive pair‐matching in randomized trials with unbiased and efficient effect estimation

“…By obviating estimation of the covariate distribution, estimators of the conditional parameter will often be less variable than estimators of the population parameter [16, 38–40]. We contrasted the unadjusted estimator with TMLE adjusting for baseline covariates.…”

“…An alternative estimand involves conditioning on the baseline covariates of the study communities [16, 38–40]: $${\psi }^F=\frac{1}{n}\sum _{i=1}^{n}E[Y_i\left(1\right)-Y_i\left(0\right)\mid W^n]$$ where Y i ( a ) denotes the counterfactual cumulative incidence under treatment level A = a for unit i . This parameter is the difference in the expected counterfactual outcomes, treating the baseline covariates of the study communities as fixed.…”

“…Greater generalizability is up to the reader and not implicitly assumed in the parameter specification. Furthermore, by obviating estimation of the covariate distribution, estimators of the conditional parameter will also often be more precise than those of the population parameter [16, 38–40]. …”

“…For ${\hat{\psi }}_{unadj}$, this is equivalent to the sample variance of the within pair differences, divided by n /2, and is commonly recommended for pair-matched randomized trials [17] even though it is known to be conservative if the conditional parameter is the target of inference [16, 38–40]. To obtain a less conservative variance estimator for ${\hat{\psi }}_{unadj}$, Abadie and Imbens [40] proposed a matching estimator, involving the variance of pairs-of-pairs with similar covariates. Our approach to reduce the true variance of the estimator and obtain a less conservative variance estimate is through adjustment with TMLE.…”

“…For brevity, we will refer to this causal parameter as the conditional average treatment effect (CATE). This parameter was initially proposed in Abadie and Imbens [38], can be interpreted as the intervention effect, given the covariates of the sample at hand, and often leads to more precise estimators [16, 39, 40]. …”