Modifications for the tukey test procedure and evaluation of the power and efficiency of multiple comparison procedures

Abstract

Multiple pairwise comparison tests of treatment means are of great interest in applied research. Two modifications for the Tukey test were proposed. The power of unilateral and bilateral Student, Waller-Duncan, Duncan, SNK, REGWF, REGWQ, Tukey, Bonferroni, Sidak, unilateral Dunnet statistical tests and the modified tests, Sidak, Bonferroni 1 and 2, Tukey 1 and 2, has been compared using the Monte Carlo method. Data were generated for 600 experiments with eight treatments in a randomized block design, of which 400 had four and 200 eight blocks. The differences between the treatment means in relation to the control were 30%, 20%, 15%, 10%, 5%. Two extra treatments did not differ from the control. A coefficient of variation of 10% and a probability Type I error of a = 0.05 were adopted. The power of all the tests decreased when the differences to the control, decreased. The unilateral and bilateral Student t, Waller-Duncan and Duncan tests showed greater number of significative differences, followed by unilateral Dunnett, modified Sidak, modified Bonferroni 1 and 2, modified Tukey 1, SNK, REGWF, REGWQ, modified Tukey 2, Tukey, Sidak and Bonferroni. There is great loss of efficiency for all tests in relation to the unilateral Student t test for each difference of the treatment to the control, when the differences between means decrease. The modified tests were always more efficient than their original ones.

Avaliação de testes estatísticos de comparações múltiplas de médias

“…Os testes de comparações múltiplas entre médias de tratamentos são de grande interesse na pesquisa aplicada (Conagin et al, 2008), quando o objetivo do trabalho é comparar tratamentos qualitativos. A utilização de um teste é realizada quando a análise de variância detecta existência de efeito significativo dos tratamentos a um determinado nível de significância, de modo que se decide rejeitar a hipótese de nulidade (pelo menos um contraste ortogonal entre tratamentos diferentes de zero).…”

A systematic review of the use of statistics in studies of restoration ecology of arid areas

“…Post Hoc tests can increase the statistical power of the analysis overall when used correctly (Conagin et al 2008). The most used test among the review studies was some form of ANOVA, which can usually be followed by some Post Hoc test.…”

“…Either way this should be a concern for anyone publishing a study of restoration ecology. Post Hocs can increase the power of an analysis and often can test us more clearly what is going on in an experiment (Conagin et al 2008). Furthermore a detailed description of how the statistical tests were carried out will make the results of the study more useful to other researchers.…”

“…The popularity of ANOVAs will likely never be challenge, but other tests such as the Mantel test could prove useful to ecologists (Dutilleul et al 2000). Post Hoc tests should also be considered where appropriate (Conagin et al 2008). More detail in explaining the choice of test and how it was preformed would also benefit many studies.…”

Analyzing one-way experiments: a piece of cake of a pain in the neck?

“…The most popular tests for mean separation that are resistant to multiple testing are for example Fisher's protected LSD, Tukey's HSD, Scheffé's, Duncan's and Neuman-Keul's tests. Researchers still work on multiple comparison procedures (e.g., Barbin,, 2006 andConagin et al, 2008).…”