Meta-analyzing dependent correlations with correction for artifacts that multiplicatively attenuate the true correlation

Cheung, Shu Fai; Chan, Darius K.-S.; Sun, Rong Wei

doi:10.3758/s13428-018-1111-y

Cited by 3 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1In practice, reliability estimates may not be available and/or other artifacts (e.g., range restriction) in addition to measurement error exist. I encourage readers to refer to relevant studies that comprehensively consider various artifacts and/or missing information on these artifacts when meta-analyzing correlations (e.g., Cheung et al, 2019; Hunter & Schmidt, 2004; Hall & Brannick, 2002; Raju et al, 1991). …”

mentioning

confidence: 99%

Meta-analysis of correlation coefficients: A cautionary tale on treating measurement error.

Zhang¹

2024

Psychological Methods

View full text Add to dashboard Cite

A scale to measure a psychological construct is subject to measurement error. When meta-analyzing correlations obtained from scale scores, many researchers recommend correcting for measurement error. I considered three caveats when correcting for measurement error in meta-analysis of correlations: (a) the distribution of true scores can be non-normal, resulting in violation of the normality assumption for raw correlations and Fisher's z transformed correlations; (b) coefficient alpha is often used as the reliability, but correlations corrected for measurement error using alpha can be inaccurate when some assumptions of alpha (e.g., tau-equivalence) are violated; and (c) item scores are often ordinal, making the disattenuation formula potentially problematic. Via three simulation studies, I examined the performance of two meta-analysis approaches-with raw correlations and z scores. In terms of estimation accuracy and coverage probability of the mean correlation, results showed that (a) considering the truescore distribution alone, estimation of the mean correlation was slightly worse when true scores of the constructs were skewed rather than normal; (b) when the tau-equivalence assumption was violated and coefficient alpha was used for correcting measurement error, the mean correlation estimates can be biased and coverage probabilities can be low; and (c) discretization of continuous items can result in biased estimates and undercoverage of the mean correlations even when tau-equivalence was satisfied. With more categories and/or items on a scale, results can improve whether tau-equivalence was met or not. Based on these findings, I gave recommendations for conducting meta-analyses of correlations.

show abstract

mentioning

confidence: 99%

Meta-analysis of correlation coefficients: A cautionary tale on treating measurement error.

Zhang¹

2024

Psychological Methods

View full text Add to dashboard Cite

show abstract

“…Footnotes 1 In practice, reliability estimates may not be available and/or other artifacts (e.g., range restriction) in addition to measurement error exist. We encourage readers to refer to relevant studies that comprehensively consider various artifacts and/or missing information on these artifacts when meta-analyzing correlations (e.g., Cheung, Chan, & Sun, 2018;Hunter & Schmidt, 2004;Hall & Brannick, 2002;Raju, Burke, Normand, & Langlois, 1991).…”

Section: 1037/met0000161mentioning

confidence: 99%

Meta-Analysis of Correlation Coefficients: A Cautionary Tale on Treating Measurement Error

Zhang¹

2021

Preprint

View full text Add to dashboard Cite

A scale to measure a psychological construct is subject to measurement error. When meta-analyzing correlations obtained from scale scores, many researchers recommend correcting measurement error. We considered three caveats when conducting meta-analysis of correlations: (1) the distribution of true scores can be non-normal, resulting in a violation of the normality assumption for raw correlations and Fisher's z transformed correlations; (2) coefficient alpha is often used as the reliability, but correlations corrected for measurement error using alpha can be inaccurate when some assumptions (e.g., tau-equivalence) of alpha are violated; and (3) item scores are often ordinal, making the disattenuation formula potentially problematic. Via three simulation studies, we examined the performance of two meta-analysis approaches with raw correlations and z scores. In terms of estimation accuracy and coverage probability of the mean correlation, results showed that (1) the true score distribution alone had slight influence; (2) when the tau-equivalence assumption was violated and coefficient alpha was used for correcting measurement error, the mean correlation estimate can be biased and coverage probability can be low; and (3) discretization of continuous items can result in under-coverage of the mean correlation even when tau-equivalence was satisfied. With more categories and/or items on a scale, results can improve when tau-equivalence was met or not. Based on these findings, we then gave recommendations when conducting meta-analysis of correlations.

show abstract

The Associations Between Physician-Patient Communication and Adjustment Outcomes of Patients and Physicians: A Systematic Review and Meta-Analysis of Correlations

Alves-Nogueira,

Góis,

Pereira

et al. 2023

Health Communication

View full text Add to dashboard Cite

Meta-analyzing dependent correlations with correction for artifacts that multiplicatively attenuate the true correlation

Cited by 3 publications

References 30 publications

Meta-analysis of correlation coefficients: A cautionary tale on treating measurement error.

Meta-analysis of correlation coefficients: A cautionary tale on treating measurement error.

Meta-Analysis of Correlation Coefficients: A Cautionary Tale on Treating Measurement Error

The Associations Between Physician-Patient Communication and Adjustment Outcomes of Patients and Physicians: A Systematic Review and Meta-Analysis of Correlations

Contact Info

Product

Resources

About