When using qualitative coding techniques, establishing inter-rater reliability (IRR) is a recognized method of ensuring the trustworthiness of the study when multiple researchers are involved with coding.

The Appelbaum and Cramer comparison of models strategy for analysis of data from nonorthogonal designs is compared with the Overall and Spiegel Method 1 general linear model analysis. Data were generated by Monte Carlo methods to include known true analysis of variance (ANOVA) main and interaction effects. In the presence of a true but nonsignificant interaction, estimates of main effect parameters derived from the Method 1 general linear model analysis were significantly closer to the true values. Greater accuracy in estimation of main effects in the presence of a significant interaction was also observed. The power of tests of significance did not appear to differ. The danger of letting observed data determine the ANOVA model and the hypotheses to be tested is emphasized.