Residual Analysis and Data Transformations: Important Tools in Statistical Analysis

Abstract

Analysis of variance (ANOVA) is a commonly used statistical analysis in agricultural experiments. Additivity, variance homogeneity, and normality are often considered prerequisites for ANOVA (Cochran, 1943; Eisenhart, 1947). The interpretation of ANOVA is valid when the random errors are independently distributed according to a normal distribution with zero mean and an unknown but fixed variance (Kempthorne, 1952; Scheffe, 1959; Steel and Torrie, 1980). Failure to meet one or more of these assumptions affects the significance levels and the sensitivity of the F test (Gomez and Gomez, 1984; Kempthorne, 1952; Little and Hills, 1978) Thus, strong deviations from one or more of the assumptions must be checked and corrected before the statistical analysis and interpretation of the results. Discrepancies of many kinds between an assumed model and the data can be detected by studying the error component or residuals (Anscombe and Tukey, 1963; Emerson and Stoto, 1983). The residuals are the deviation from the observed and the predicted values according to the assumed model. If the assumptions about the validity of the model are valid, a residual plot (scatter plot between the residuals and the predicted values) will have a random distribution. If the residual plot has an unexplained systematic pattern, then the ANOVA model is not appropriate. Residual plots can be used to detect the violation of assumptions in ANOVA, such as variance heterogeneity (unequal variance], auto-correlated error (nonindependence), and the presence of outliers. Thus, it is crucial to examine the residuals before interpreting the data. Violation of assumptions in ANOVA Nonadditivity, variance heterogeneity, and nonnormality. The additivity requirement implies that the block and treatment effects should be additive. For example, in a randomized complete-block design, the differ

Comparing phosphorus mobilization strategies using Aspergillus niger for the mineral dissolution of three phosphate rocks

“…Using the SAS Proc Univariate procedure (Fernandez 1992; Bowley 1999), the residuals of the model were plotted against the predicted residual values and the relevant treatment(s) and analysed for homogeneity. The residuals were also tested for normality as determined by the Shapiro–Wilk statistic.…”

A Hedonic Analysis of Apple Prices and Product Quality Characteristics in British Columbia

“…The relationship between the variances of the residuals and the predicted values is used to determine the power transformation required to produce homogeneity of variance (Fernandez 1992). This analysis shows that the 1/4 power of price would produce homogeneous variances.…”

Transgene integration patterns and expression levels in transgenic tissue lines of Picea mariana, P. glauca and P. abies

“…GUS expression level was transformed to its natural logarithm to stabilize the variance (Box-Cox transformation, Fernandez 1992). Mean GUS activity level and its 95% confidence limits were computed on the logarithmic scale and back-transformed for presentation in the tables.…”