In paired randomized experiments units are grouped in pairs, often based on covariate information, with random assignment within the pairs. Average treatment effects are then estimated by averaging the within-pair differences in outcomes. Typically the variance of the average treatment effect estimator is estimated using the sample variance of the within-pair differences. However, conditional on the covariates the variance of the average treatment effect estimator may be substantially smaller. Here we propose a simple way of estimating the conditional variance of the average treatment effect estimator by forming pairs of-pairs with similar covariate values and estimating the variances within these pairs-of-pairs. Even though these within-pairs-of-pairs variance estimators are not consistent, their average is consistent for the conditional variance of the average treatment effect estimator and leads to asymptotically valid confidence intervals.