A Monte Carlo comparison of studentized bootstrap and permutation tests for heteroscedastic two-sample problems

Janssen, Arnold; Pauls, Thorsten

doi:10.1007/bf02741303

Cited by 34 publications

(28 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most cases, the power of the permutation t-test was found to be higher compared with exact Wilcoxon rank sum test in current study. Similar to our study, a number of other studies described power advantage with permutation test compared with Wilcoxon rank sum test [24,37,45]. In contrary, Wilcoxon rank sum test yielded remarkably powerful results than both t-test and permutation test in a study conducted by Weber and Sawilowsky [9].…”

Section: Discussionsupporting

confidence: 85%

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method

Dwivedi

Mallawaarachchi

Alvarado

2017

Statistics in Medicine

226

138

View full text Add to dashboard Cite

Experimental studies in biomedical research frequently pose analytical problems related to small sample size. In such studies, there are conflicting findings regarding the choice of parametric and nonparametric analysis, especially with non-normal data. In such instances, some methodologists questioned the validity of parametric tests and suggested nonparametric tests. In contrast, other methodologists found nonparametric tests to be too conservative and less powerful and thus preferred using parametric tests. Some researchers have recommended using a bootstrap test; however, this method also has small sample size limitation. We used a pooled method in nonparametric bootstrap test that may overcome the problem related with small samples in hypothesis testing. The present study compared nonparametric bootstrap test with pooled resampling method corresponding to parametric, nonparametric, and permutation tests through extensive simulations under various conditions and using real data examples. The nonparametric pooled bootstrap t-test provided equal or greater power for comparing two means as compared with unpaired t-test, Welch t-test, Wilcoxon rank sum test, and permutation test while maintaining type I error probability for any conditions except for Cauchy and extreme variable lognormal distributions. In such cases, we suggest using an exact Wilcoxon rank sum test. Nonparametric bootstrap paired t-test also provided better performance than other alternatives. Nonparametric bootstrap test provided benefit over exact Kruskal-Wallis test. We suggest using nonparametric bootstrap test with pooled resampling method for comparing paired or unpaired means and for validating the one way analysis of variance test results for non-normal data in small sample size studies. Copyright © 2017 John Wiley & Sons, Ltd.

show abstract

Section: Discussionsupporting

confidence: 85%

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method

Dwivedi

Mallawaarachchi

Alvarado

2017

Statistics in Medicine

226

138

View full text Add to dashboard Cite

show abstract

“…WARNING: Relatively large samples are needed for robustness (Wilcox, 2010: 91), at least n > 10 (Chernick, 1999). WARNING: With heavy tails the bootstrap should be avoided (Janssen & Pauls, 2005). WARNING: The Bt is sensitive to outliers (Good, 2006 This test was developed by Keselman et al (2002Keselman et al ( , 2004, and involves (a) a transformation of the data for skewness (Hall, 1992), (b) Winsorized trimming (10-15%), (c) Welch adjustment and (d) bootstrapping; free R programs are available.…”

Section: Appendix a (Continued )mentioning

confidence: 99%

The t test and beyond: Recommendations for testing the central tendencies of two independent samples in research on speech, language and hearing pathology

Rietveld

Hout

2015

Journal of Communication Disorders

View full text Add to dashboard Cite

The main messages are (a) that researchers should present more relevant features of their data (means, medians, SD, skewness, tailedness, outliers etc.), (b) not routinely use conventional non-parametric tests like Wilcoxon-Mann-Whitney test in case one or more of the assumptions of t tests are not met, and (c) should consider using less conventional, but robust statistics which have been developed and tested in the last decades.

show abstract

“…These Welch tests are, however, known to be rather conservative procedures if the underlying distribution is nonsymmetric . Thus, to enhance their power, Janssen proposed a studentized permutation test in the Welch‐type statistic ( T w ) that is based on randomly permuting the pooled sample; see also the works of Chung and Romano and Pauly et al Similar to φ t , p the resulting Welch permutation test, φ w , p = 1{ T w > c w , p (1 − α 2 )} (in the one‐sided case) is an exact procedure if the data is exchangeable and in general asymptotically correct.…”

Section: Semiparametric Proceduresmentioning

confidence: 99%

Multiplication‐combination tests for incomplete paired data

Amro¹,

Konietschke

Pauly

2019

Statistics in Medicine

View full text Add to dashboard Cite

We consider statistical procedures for hypothesis testing of real valued functionals of matched pairs with missing values. In order to improve the accuracy of existing methods, we propose a novel multiplication combination procedure.Dividing the observed data into dependent (completely observed) pairs and independent (incompletely observed) components, it is based on combining separate results of adequate tests for the two sub data sets. Our methods can be applied for parametric as well as semiparametric and nonparametric models and make use of all available data. In particular, the approaches are flexible and can be used to test different hypotheses in various models of interest. This is exemplified by a detailed study of mean-as well as rank-based approaches under different missingness mechanisms with different amount of missing data. Extensive simulations show that in most considered situations, the proposed procedures are more accurate than existing competitors particularly for the nonparametric Behrens-Fisher problem. A real data set illustrates the application of the methods.

show abstract

A Monte Carlo comparison of studentized bootstrap and permutation tests for heteroscedastic two-sample problems

Cited by 34 publications

References 19 publications

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method

Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method

The t test and beyond: Recommendations for testing the central tendencies of two independent samples in research on speech, language and hearing pathology

Multiplication‐combination tests for incomplete paired data

Contact Info

Product

Resources

About