Comparision Mann-Whitney U Test and Students’ t Test in Terms of Type I Error Rate and Test Power: A Monte Carlo Sımulation Study

Abstract

ÖzetParametrik t testinin varsayımları karşılanmadığı durumda alternatif olarak Mann-Whitney U testi kullanılmaktadır. Varsayımlar yerine getirildiğinde t testinin daha güvenli olduğu bilinmektedir. Bu çalışmanın amacı kitlenin normal dağılımlı olması durumunda ve çeşitli örnek hacimlerinde MWU ile t testinin performanslarını karşılaştırmaktır. Kitleden örnek seçme için Monte Carlo simülasyonu yapıldı. Ortalamalar arası fark δ=0, δ=0.5σ, δ=1σ ve varyans oranları 1:1, 1:4, 1:9 olan kitlelerin ikili kombinasyonları kullanıldı. Birinci tip hata ve test gücünü hesaplamak için her bir kombinasyon için 20000 Monte Carlo denemesi yapıldı. Her bir kombinasyon ve farklı örnek hacimleri için çekilen örneklere her iki test uygulandı, red edilen sıfır hipotezi sayısı kaydedildi. Küçük örnek (n≤30) hacminde MWU testinin daha az hata verdiği, H 0 hipotezinin ret veya kabul edilmesinde MWU ile t testlerinin %97 oranında aynı sonucu verdikleri görüldü. Eşit olmayan varyans durumunda özellikle örnek hacmi arttıkça MWU testinin I.Tip hata oranını koruyamadığı görüldü. Comparision Mann-Whitney U Test and Students' t Test in Terms of AbstractThe nonparametric Mann-Whitney U test is an alternative to parametric Student's t-test, when the assumptions of t-test cannot be respected. However as one understood, it is more reliable to use the ttest if its assumptions can be met. The purpose of this study is to analyze the performances of the Mann-Whitney U test and Student's t test in cases of normal populations and various sample sizes. A Monte Carlo simulation study is cariried out via data which were produced from standard normal distribution. Dual combinations of populations were used, with a difference between the means of δ=0, δ=0.5σ, δ=1σ and, variance ratios of 1:1, 1:4, 1:9. To compute Type I error rate and test power, 20000 trial were made for each condition. For each combination, two samples with various sample sizes were randomly selected; both tests were applied, and the number of rejected null hypotheses was recorded. The MWU test gave fewer errors in small samples (n≤30). MWU and t tests gave the same result at 98% rate in rejecting or accepting the H 0 hypothesis. In case of unequal variance, as sample size increased, the type I error ratio increased, particularly for the U test; both tests gave the same result in approximately 97% of simulations.

Comparison of Student – t, Welch s t, and Mann – Whitney U Tests in Terms of Type I Error Rate and Test Power

“…For all combinations, Welch's t-test yielded results very close to the type I error rate determined at the beginning of the experiment 5% and may be definable as conservative. These findings are consistent with the studies of Ahad and Yahaya (2014), Bindak (2014), andRuxton (2006).…”

“…The reason for the Mann-Whitney U test having such a trend could be the assumption that the populations from which we took our groups were normally distributed. This result is similar to the study conducted by Bindak (2014). While the variances were homogeneous for the Student's t-test, a desired power value of 80% could be achieved with sample sizes of 30, 25, and 15 for all differences in standard deviations, respectively.…”

“…Across all the combinations of variance ratios and sample sizes we tested, Welch's t-test yielded results that satisfied the Bradley's criterion of robustness. Similar to Bindak's (2014) study, cases such as positive and negative associations were also tried. Based on the results of the large variances and large sample sizes simulations (positive association), Student -t test did not satisfy Bradley's criterion.…”

Comparison of type I error rates of Wald-Wolfowitz, Kolmogorov-Smirnov, and Mann-Whitney tests in small, equal, and large sample volumes

“…Genel olarak, etkileşim etkilerini test ederken I. tip hata oranını rutin olarak yükseltmenin tavsiye edilmediği, bunun yerine araştırmacılara etkileşim etkilerini önceden belirleme ve örneklem büyüklüğü hesaplamalarında bunları dikkate alma önerisi sunulmuştur. Bindak (2014), parametrik t testinin varsayımlarının karşılanmadığı durumlarda alternatif olarak kullanılan Mann-Whitney U testi ile t testinin performanslarını karşılaştırmayı amaçlamıştır. Kitlenin normal dağılıma sahip olduğu durumu ele alan çalışmada, çeşitli örnek hacimleri ve farklı varyans oranları kullanılarak Monte Carlo simülasyonu gerçekleştirilmiştir.…”

The power and type I error of Wilcoxon-Mann-Whitney, Welch's t, and student's t tests for Likert-type data

“…However, Likert scale data (i.e., the data that are the sum or mean of Likert items) can be analyzed under the assumption that they are interval data (Boone & Boone, 2017). For interval data, it is well known that the Student's t-test has better statistical power than the U-test in many cases when comparing means (Boneau, 1962;Glass et.al., 1972;Zimmerman & Zumbo, 1990;Bindak, 2014). Some researchers claim that this approach is also valid for data such as the Likert scale (Norman, 2010).…”