An effect size index for comparing two independent alpha coefficients

Abstract

Since Cronbach proposed the alpha coefficient in 1951, researchers have contributed to the derivation of its sampling distribution and the testing of related statistical hypotheses. Yet, there has been no research on effect size index relevant to coefficient alpha to our knowledge. Considering the importance of effect size in understanding quantitative research findings, we therefore developed an effect size index Delta for the comparison of two independent alphas with equal test length based on the asymptotic distribution of (1/2)ln(1 - alphahat) under the assumptions of normality and compound symmetry. Simulations indicated that the index was applicable when the sample size was at least 100. The robustness of the derived index when the required assumptions were violated was also explored. It is suggested that the index should be applicable in most cases of unequal test lengths and could be extended to non-normally distributed component scores. Moreover, a small simulation was conducted to explore the behaviour of Delta with correlated errors, a frequently studied situation violating the assumption of compound symmetry. The proposed index was found to be robust unless a large number of highly correlated errors were present in the data.

Rugby versus Soccer in South Africa: Content familiarity contributes to cross-cultural differences in cognitive test scores

“…Most values were acceptable to high. The highest values were found for the attention test, most probably due to the high score variation for (Hakstian & Whalen, 1976) and their effect sizes (Liu & Weng, 2009) were computed. Seven out of ten comparisons showed a significant difference.…”

Disaggregated Effects of Device on Score Comparability

“…Observed test score reliability was near .70 for reading and science and above .85 for math and was similar for both device conditions (effect sizes of .089, –.042, and –.052, for reading, math, and science, respectively; Liu & Weng, ). The higher observed reliability of test scores in math was likely due to the higher mean item–total correlations on the test form.…”

Noncompliant responding: Comparing exclusion criteria in MTurk personality research to improve data quality

“…Effect sizes were calculated with negative values. Interpretation of effect size Δ is equivalent to Cohen's d (Liu & Weng, 2009 participants (17.13% of the sample) were flagged as giving noncompliant responses by at least one method. Of these participants, 91 were identified by two or more approaches and 77 respondents (7.85%) were uniquely identified by only one approach.…”

“…Effect sizes were calculated with negative values. Interpretation of effect size Δ is equivalent to Cohen's d (Liu & Weng, 2009). ⁎ p < .05.…”