Pooling ANOVA Results From Multiply Imputed Datasets

Abstract

Abstract. The analysis of variance (ANOVA) is frequently used to examine whether a number of groups differ on a variable of interest. The global hypothesis test of the ANOVA can be reformulated as a regression model in which all group differences are simultaneously tested against zero. Multiple imputation offers reliable and effective treatment of missing data; however, recommendations differ with regard to what procedures are suitable for pooling ANOVA results from multiply imputed datasets. In this article, we compared several procedures (known as D1, D2, and D3) using Monte Carlo simulations. Even though previous recommendations have advocated that D2 should be avoided in favor of D1 or D3, our results suggest that all procedures provide a suitable test of the ANOVA’s global null hypothesis in many plausible research scenarios. In more extreme settings, D1 was most reliable, whereas D2 and D3 suffered from different limitations. We provide guidelines on how the different methods can be applied in one- and two-factorial ANOVA designs and information about the conditions under which some procedures may perform better than others. Computer code is supplied for each method to be used in freely available statistical software.

Dissecting the Genetic Architecture of Shoot Growth in Carrot (Daucus carotaL.) Using a Diallel Mating Design

“…Imputation has advantages over alternative methods to address missing data in diallel experiments in that it is relatively simple to implement, makes use of all available data for a given trait, and replaces missing data with plausible estimates to avoid reductions in sample size. However, there are several caveats and compromises regarding multiple imputation, namely, that there are inadequate or vague diagnostics and, although simple in principle, methods to pool multi-factor ANOVA results are often vague, or are not widely accessible ( van Ginkel and Kroonenberg 2014 ; Grund et al 2016 ). In this study, we demonstrate the application of existing, relatively straightforward, methods to pool results for diallel analysis across multiple environments.…”

Significance Tests and Estimates for R2 for Multiple Regression in Multiply Imputed Datasets: A Cautionary Note on Earlier Findings, and Alternative Solutions

“…Barnard and Rubin (1999) and Reiter (2007) developed improved error degrees of freedom for these combination rules. Simulation studies (Barnard & Rubin, 1999;Grund, L€ udtke, & Robitzsch, 2016;Liu & Enders, 2017;Reiter, 2007;Schafer, 1997) have shown that these combination rules generally give type-I error rates close to the theoretical type-I error rates.…”

“…For testing several parameters for significance simultaneously, several solutions are available Meng & Rubin, 1992;Rubin, 1987). Of these solutions, the most promising one Rubin, 1987) according to several simulation studies (Grund et al, 2016;Liu & Enders, 2017;Reiter, 2007) is a set of formulas that are multivariate extensions of Equations (3)-(8).…”

Multiple Imputation of Multilevel Missing Data

“…In addition to Rubin's rules (1987), the package implements the procedures commonly referred to as D 1 Reiter, 2007), D 2 (Li, Meng, Raghunathan, & Rubin, 1991), and D 3 (Meng & Rubin, 1992), which can be used for testing a variety of statistical hypotheses that potentially involve multiple parameters simultaneously (e.g., model comparisons Li, Meng, et al (1991) and Meng and Rubin (1992), or on variations thereof (Licht, 2010), but clear recommendations have not yet been made in the literature (see also Consentino & Claeskens, 2010;Grund, Lüdtke, & Robitzsch, 2016b).…”