Combining Paired and Two-Sample Data Using a Permutation Test

Abstract

This paper presents a permutation test for the incomplete pairs setting. This situation arises in both observational and experimental studies when some of the data are in the form of a paired sample and the rest of the data comprise two independent samples. The proposed method uses the data from the two types of samples to test the difference between the mean responses. Our test statistic combines the observed mean difference for the complete pairs with the difference between the two means of the independent samples. The randomizations are carried out as is typically done with standard permutation tests for paired and independent samples. We show by a simulation study that our statistic performs well in comparison to other methods.

Permutation tests for mixed paired and two-sample designs

“…Dubnicka's [3] weighted statistic also had higher power than the unweighted statistic as the correlation increased. Einsporn and Habtzghi's [4] statistic had power similar to or slightly higher than Dubnicka's [3] weighted statistic for data from a normal distribution. When the data came from an exponential distribution, Dubnicka's [3] weighted statistic always performed better.…”

“…Their proposed test usually had marginally higher power than older tests proposed by Bhoj [2], Lin and Stivers [6], Maritz [8], and Dubnicka et al [3] for small sample sizes and low correlation under a normal distribution. Bhoj's statistic [2] had the highest power of the parametric procedures considered and was often better than Einsporn and Habtzghi's [4] permutation test for high correlation and large sample sizes. The weighted nonparametric statistic proposed by Dubnicka et al [3] almost always had highest power when the data were exponential.…”

“…Einsporn and Habtzghi [4] proposed a permutation test that was a combination of standard permutation tests for means of the paired and independent samples. Unlike the permutation scheme proposed by Maritz, they chose to permute the incomplete pairs such that the sample size for each variable remained fixed in all arrangements.…”

“…The nonparametric tests generally performed better in the limited simulations of Einsporn and Habtzghi [4], but a wider variety of conditions, such as heavier-tailed distributions and different sample size combinations, may need to be considered to better compare the different methods. Einsporn and Habtzghi [4] found that the permutation test on means tended to have higher power than Bhoj's parametric asymptotic test on means [2]. This suggests that a permutation version of the rank statistics may also perform better than the asymptotic versions, especially for small to moderate sample sizes.…”

“…This new statistic generally had higher power when there were more incomplete than complete pairs, especially under a normal distribution. Einsporn and Habtzghi [4] proposed a permutation test on the weighted mean difference that performed with higher power than the weighted test of Dubnicka et al [3] for data generated from a normal distribution but not from an exponential distribution. Those were also the only two distributions considered by Einsporn and Habtzghi along with only two sets of sample sizes.…”