Cohen’s d needs to be readily interpretable: Comment on Shieh (2013)

Abstract

Shieh (2013) discussed in detail δ*, a proposed standardized effect size measure for the two-independent-groups design with heteroscedasticity. Shieh focused on inference-notably, the large challenge of calculating confidence intervals for δ*. I contend, however, that the standardizer chosen for δ*, meaning the units in which it is expressed, is appropriate for inference but causes δ* to be inconsistent with conventional Cohen's d. In addition, δ* depends on the relative sample sizes in the particular experiment and, thus, lacks the generality that is highly desirable if a standardized effect size is to be readily interpretable and also usable in meta-analysis. In the case of heteroscedasticity, I suggest that researchers should choose as standardizer for Cohen's δ the best available estimate of the SD of an appropriate population, usually the control population, in preference to δ* as discussed by Shieh.

Health‐related quality of life in paediatric haemophilia B patients treated with rIX‐FP

“…18,19 In the present study, the MID from baseline to EOS was determined using a distribution-based approach applying Cohen's d effect size calculation. 17 The Cohen's d effect size is a standardized measure of change obtained by dividing the difference in mean scores from baseline to EOS by the standard deviation of baseline scores. This can be mathematically represented as: 20 Cohen has proposed the following benchmarks for interpreting effect sizes d (d = 0.2, small effect size; d = 0.5, medium effect size; d = 0.8, large effect size).…”

“…The magnitude of minimal important differences (MID) on a group level, defined by Guyatt as “smallest difference in score in the domain of interest that patients perceive as important and that would lead the clinician to consider a change in the patient's management’’, can be determined by distribution‐based and anchor‐based approaches . In the present study, the MID from baseline to EOS was determined using a distribution‐based approach applying Cohen's d effect size calculation …”

Sustained impact of the “Healthy Habits, Healthy Girls – Brazil” school-based randomized controlled trial for adolescents living in low-income communities

“…A d of 1 indicates the two groups' means differ by one standard deviation; a d of 0.5 indicates that two groups' means differ by half a standard deviation; and so on. Cohen suggested that d = 0.2 be considered a “small” effect size, 0.5 represents a “medium” effect size and 0.8 a “large” effect size, e.g., if two groups' means don't differ by 0.2 standard deviations or more, the difference is trivial, even if statistically significant ( Cumming, 2013 ). To minimize the problem of participation bias, all statistical analyses followed the intention-to-treat principle ( Gupta, 2011 ), significance was set at p < 0.05 and the confidence interval was estimated to be 95%.…”

Response of Sclerostin and Bone Turnover Markers to High Intensity Interval Exercise in Young Women: Does Impact Matter?

“…Effect sizes (ES), including partial eta squared f (partial η 2 ) for ANOVA and Cohen's d (mean difference/standard deviation pretest) for significant pairwise comparisons were also calculated [ 31 ]. ES were then interpreted based on the Cohen criteria: 0.01=small, 0.06=moderate, 0.14=large effect for partial η 2 , and 0.2=small, 0.5=medium, and 0.8=large effect for Cohen's d [ 32 ].…”