The accuracy of effect-size estimates under normals and contaminated normals in meta-analysis

Abstract

This article evaluates the accuracy of effect-size estimates for some estimation procedures in meta-analysis. The dilemma of which effect-size estimate is suitable is still a problem in meta-analysis. Monte Carlo simulations were used to generate random variables from a normal distribution or contaminated normal distribution for primary studies. The primary studies were hypothesised to have equal variance under different population effect sizes. The primary studies were also hypothesised to have unequal variance. Meta-analysis was done on the simulated hypothesized-primary-studies. The effect sizes for the simulated design of the primary studies were estimated using Cohen's d , Hedges' g , Glass' △, Cliff's delta d and the Probability of Superiority. Their corresponding standard error and confidence interval were computed and a comparison of an efficient estimator was done using statistical bias, percentage error and confidence interval width. The statistical bias, percentage error and confidence interval width pointed to Probability of Superiority as an accurate effect size estimate under contaminated normal distribution, and Hedges' g as the most accurate effect size estimates compared to Cohen's d and Glass' △ when equal variance assumptions are violated. This study suggests that the accuracy of effect size estimates depends on the details of the primary studies included in the meta-analysis.

Effects of Low-Frequency Repetitive Transcranial Magnetic Stimulation on Language Recovery in Poststroke Survivors With Aphasia: An Updated Meta-analysis

“…Hedges’ g for effect size was given by using the mean change in performance from baseline in the control group with respect to the experimental group and dividing the difference by the pooled SD of the change from baseline. 18,19 The pooled effect size was interpreted as small (≤0.2), moderate (0.5), or large (≥0.8) based on the Cohen criteria. 20…”

A meta-analysis of the effects of problem-based learning on students' creative thinking in mathematics

“…the degrees of freedom used in estimating ℎ and is the Cohen's calculated using the equation and the variance of the Hedge's g effect size = 2 where is the variance of Cohen's d (Marfo & Okyere, 2019). In addition to the calculations of the weighted mean effect size of Hedges' g with its standard errors and 95% confidence intervals around each mean, CMA was also used to test homogeneity by calculating Q, p, and 2 (Hillmayr et al, 2020).…”

Effect on dynamic postural stability after cold therapy application to the knee joint: a pilot randomized controlled trial

“…We used the sample size of 12 in each group, as it was sufficient for a pilot study [8]. The effect size (ES) was calculated by Cohen's d by using the following formula [9]:…”