Testing for normality in the multi-group problem: is this a good practice?

Abstract

SummaryOne of the validity conditions of classical test statistics (e.g., the ANOVA F-test) is that data be normally distributed in the populations. When this derivational assumption does not hold classical test statistics can be prone to falsely rejecting too often and/or fail to reject when the null hypothesis is false. Thus, many authors have recommended that researchers routinely check that the assumption of normality is satisfied. The findings regarding the power of normality tests are mixed. That is, some authors believe that tests for normality lack power to detect non-normality in the data; others however, have found that some tests for non-normality are quite effective in detecting non-normal data. The purpose of our study was to re-examine this question but in the multi-group problem. We found that two of the goodness-of-fit statistics, the Anderson-Darling and Cramer-von Mises tests were quite effective in detecting non-normality over a large number of non-normal data conditions with sample sizes that were not unreasonably large, particularly if a wiser choice is made regarding the level of significance (e.g., α = 15 or α = 20). We also demonstrate how overall Type I error control can be maintained over the multiple tests for normality with a sequentially-rejective Bonferroni procedure.

Generalized Linear Model Analyses for Treatment Group Equality when Data are Non-Normal

“…Thus, GLM model analyses appear to be very versatile tools for examining equality of treatment group effects for data that are not normal in form. However, there is limited published information regarding the effectiveness of the procedure for dealing with non-normal data (see e.g., Schoder et al, 2006;Keselman et al, 2013;2014); that is, the effectiveness of the procedures for dealing with non-normal data will depend, in part, on how good are the tests for normality, the Information Criteria, and fit in identifying the correct form of the distribution of the data (some published studies report unfavorable results -see e.g., Rochon & Kieser, 2011;Schoder et al, 2006). As a result, if the analyses/procedures cannot identify the correct shape for the distribution(s) of the data to be specified in the link function, GLM analyses will give erroneous conclusions with regard to the test for equality of treatment group effects.…”

“…Choices for non-normal distributions are modifications from Schoder et al (2006), Zimmerman (2010), and Keselman et al (2013;2014). These authors investigated a normal distribution with a single outlier, a normal distribution with 10% of the data containing outliers, skewed distributions with varying skewness, and an ordinal 5-point Likert scale with varying multivariate probabilities (common they state in psychological and medical investigations).…”

“…Finally, like Schoder et al (2006), and Keselman et al (2013;2014), a 5-point Likert scale was simulated; such data is frequently gathered in medical, psychological (e.g., from clinical, personality, and social psychology) and other behavioral science investigations. As well, there has been much discussion in the medical and social science literatures regarding the analysis of outcomes with 5 -point Likert data and classical methods of analysis (see e.g., Roberson, Shema, Mundfrom, & Holmes, 1995;Jameson, 2004;Nanna & Sawilowsky, 1998).…”

“…For example, in the one-group pretest-posttest-design, Schoder et al (2006) found that the Kolmogorov-Smirnov (K-S) goodness of fit test did not have enough power to detect non-normal distributions unless sample size was greater than 100. On the other hand, in the same design, Keselman et al (2014) found that the Anderson-Darling (A-D) goodness of fit test did have sufficient power (≥ 0.80) if researchers set the level of significance at values greater than 0.05 (e.g., α = 0.15 or α = 0.20). Again comparing goodness of fit statistics for detecting non-normal distributions in the multi-group (3) problem, Keselman et al (2014) also found that the A-D test provided good power to detect non-normal distributions even while adopting familywise Type I error control over the three tests of normality with overall significance controlled at 0.15 and 0.20.…”

“…Evidence regarding the accuracy of these tests is varied (e.g., see Dufour, Farhat, Gardiol, & Khalaf, 2010;Kowalchuk, Keselman, Wilcox, & Algina, 2006;Keselman, Othman, & Wilcox, 2013;2014;Rochon & Kieser, 2011;Schoder, Himmelmann, & Wilhelm, 2006). For example, in the one-group pretest-posttest-design, Schoder et al (2006) found that the Kolmogorov-Smirnov (K-S) goodness of fit test did not have enough power to detect non-normal distributions unless sample size was greater than 100.…”