Bias and Precision of Alternate Estimators in Meta-Analysis: Benefits of Blending Schmidt-Hunter and Hedges Approaches

Abstract

We describe a new estimator (labeled Morris) for meta-analysis. The Morris estimator combines elements of both the Schmidt-Hunter and Hedges estimators. The new estimator is compared to (a) the Schmidt-Hunter estimator, (b) the Schmidt-Hunter estimator with variance correction for the number of studies (“k correction”), (c) the Hedges random-effects estimator, and (d) the Bonett unit weights estimator in a Monte Carlo simulation. The simulation was designed to represent realistic conditions faced by researchers, including population random-effects distributions, numbers of studies, and skewed sample size distributions. The simulation was used to evaluate the estimators with respect to bias, coverage of the 95% confidence interval of the mean, and root mean square error of estimates of the population mean. We also evaluated the quality of credibility intervals. Overall, the new estimator provides better coverage and slightly better credibility values than other commonly used methods. Thus it has advantages of both commonly used approaches without the apparent disadvantages. The new estimator can be implemented easily with existing software; software used in the study is available online, and an example is included in the appendix in the Supplemental Material available online.

Perceived Workplace Gender Discrimination and Employee Consequences: A Meta-Analysis and Complementary Studies Considering Country Context

“…The 80% credibility intervals were calculated using normalρ true^ and the standard deviation of normalρ true^ ; 95% confidence intervals were calculated using true r ¯ and standard error based on sampling error variance σ e 2 when population effect size variance is zero (i.e., homogeneous) or using true r ¯ and standard error based on the residual variance of correlations after removing sampling error variance (i.e., heterogeneous) (Whitener, 1990). b Countries with a median score in our sample for the moderator variable were included in the low category. c Hunter and Schmidt (2004) method cannot compute SD normalρ true^ because it overestimates variance due to sampling error when N is small (Brannick & Hall, 2001). …”

“… c Hunter and Schmidt (2004) method cannot compute SD normalρ true^ because it overestimates variance due to sampling error when N is small (Brannick & Hall, 2001). …”

The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews

“…The motivation for this blend of techniques is clear: each has its advantages (Wiernik & Dahlke, 2020). However, procedures have been refined and, consequently, we contrast Stahl et al's results with a modern technique that better accomplishes their aim: Morris estimators (Brannick, Potter, Benitez, & Morris, 2019).…”

Meta-analysis in vocational behavior: A systematic review and recommendations for best practices

“…Following the strict advice of Schmidt-Hunter-style meta-analysis, one does not have a choice between fixed effect versus random effects estimators, as the Schmidt-Hunter model is inherently a random effects model (see Schmidt et al, 2009, p. 100). Importantly, moving beyond such strict advice, hybridized approaches that blend Schmidt-Hunter and Hedges-Olkin estimation procedures are indeed possible, and in some cases have been advocated (e.g., Brannick, Potter, Benitez, & Morris, 2019). Estimates of between-study variability in the Schmidt-Hunter tradition typically adopt a different notation, standard deviation of rho (SDρ), to represent the non-artefactual variance in effect size estimates across studies.…”