Estimating Group-Level Relationships: General Recommendations and Considerations for the Use of Intraclass Correlation Coefficients

Burke, Maura I.; Landis, Ronald S.; Burke, Michael J.

doi:10.1007/s10869-016-9464-7

Cited by 11 publications

(5 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This model and procedures hold considerable promise for studying other interrater agreement problems as well as other phenomena in group-level studies. For instance, for primary group-level studies and meta-analytic studies with group-level data, Bliese (1998) and Burke, Landis, and Burke (2016), respectively, argued for the use of ICC(2) values in correcting group-level correlations. Although these authors demonstrated the potential usefulness of such corrections with simulated and empirical data, their procedures did not allow for a more complete modeling of group level data and assessments of the accuracy of such reliability corrections.…”

Section: Discussionmentioning

confidence: 99%

Central tendency and matched difference approaches for assessing interrater agreement.

Burke

Cohen²,

Doveh³

et al. 2018

Journal of Applied Psychology

Self Cite

View full text Add to dashboard Cite

In Study 1 of this two-part investigation, we present a "central tendency approach" and procedures for assessing overall interrater agreement across multiple groups. We define parameters for mean group agreement and construct bootstrapped confidence intervals around the mean population parameters for r, AD, and ICC(1). In Study 2, we extend assessments of overall interrater agreement by developing a "matched difference approach" and procedures for assessing real versus pseudo agreement in a sample of groups. Here, we use random group resampling and the matched difference between assessments of the respective r, AD, and ICC(1) values for actual and pseudo groups, with the establishment of bootstrapped confidence intervals around such differences. In both studies, we employ simulated and real data to demonstrate the accuracy and practical utility of the new procedures for assessing agreement with respect to groups. Notably, to generate simulated data for Studies 1 and 2, we developed a new underlying model for multilevel data and procedure for data generation, and we discuss its potential utility for enhancing research in group-level studies. Moreover, we discuss, relative to current practices, how and why the new inference procedures provide information about mean interrater agreement in the population, which can improve data aggregation decisions and interpretations of findings from group-level studies. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

show abstract

Section: Discussionmentioning

confidence: 99%

Central tendency and matched difference approaches for assessing interrater agreement.

Burke

Cohen²,

Doveh³

et al. 2018

Journal of Applied Psychology

Self Cite

View full text Add to dashboard Cite

show abstract

“…In case of the multiple correlations from the same study were combined, the Mosier reliabilities of the composite measures were also computed (Schmidt & Hunter, 2015). In those few cases in which no reliabilities were reported, this study took the average reliability of the same variable from the set of primary studies, that is, an accepted method when conducting meta-analyses at the group level of analysis (M. I. Burke, Landis, & Burke, 2017).…”

Section: Methodsmentioning

confidence: 99%

“…In those few cases in which no reliabilities were reported, this study took the average reliability of the same variable from the set of primary studies, that is, an accepted method when conducting meta-analyses at the group level of analysis (M. I. Burke, Landis, & Burke, 2017). The average of the available alpha coefficients and ICC(2) values ranged from .78 to .93 and .63 to .76, respectively, for task-focused leadership and .78 to .93 and .75 to .77, respectively, for person-focused leadership.…”

Section: Meta-analytic Proceduresmentioning

confidence: 99%

Differential Relationship of Person- and Task-Focused Leadership to Team Effectiveness: A Meta-Analysis of Moderators

Pratoom

2018

Human Resource Development Review

View full text Add to dashboard Cite

Building on previous findings of C. S. Burke et al. and Ceri-Booms et al., this meta-analytic review aims to investigate the moderating impact of team size, interdependence, power distance culture, effectiveness criteria, and study settings on the differential relationship of person-and task-focused leadership with team effectiveness. Based on 52 studies with 4,958 teams, results suggested that the differential effects of leadership were moderated by effectiveness criteria and the interaction between team characteristics and culture in societies. In high power distance societies, the relationship between person-focused leadership and team effectiveness was stronger in small teams that were characterized by high interdependence. In low power distance societies, the larger effect size of a personfocused leader was found in small teams, while task-focused leadership emerged as a stronger predictor of team effectiveness than person-focused leadership in large teams. Results can be used to guide leader development programs needed to enhance team effectiveness.

show abstract

“…Given that employees were nested in 26 countries, we evaluated the need for multilevel modeling by calculating the intraclass correlation (ICC(1) and ICC(2)) for the dependent variables (Burke, Landis, & Burke, ). The ICC(1) value is interpreted as the proportion of variance explained by group membership (i.e., country), and it was calculated using the following Bartko equation (see Bliese, ): ICC(1) = (MSB – MSW) / (MSB + ( n – 1) × MSW), where MSB is the between‐group mean square, MSW is the within‐group mean square, and n is the average number of members within groups.…”

Section: Methodsmentioning

confidence: 99%

Autonomy as a key resource for women in low gender egalitarian countries: A cross‐cultural examination

Halliday

Paustian‐Underdahl

Ordóñez

et al. 2017

Human Resource Management

View full text Add to dashboard Cite

Relying on Conservation of Resources theory and a sample of 23,439 workers in 26 countries, we develop and test a multilevel moderated mediation of the effects of perceived job autonomy on work–life balance, engagement, and turnover intentions, depending on employee gender and country‐level gender egalitarianism (GE), and indirectly through stress. We find that perceived job autonomy relates to these outcomes indirectly through stress such that these effects are stronger for women in lower GE countries compared with women in higher GE countries and are not significant for men. Practical implications for engaging and retaining global female talent are discussed.

show abstract

Estimating Group-Level Relationships: General Recommendations and Considerations for the Use of Intraclass Correlation Coefficients

Cited by 11 publications

References 65 publications

Central tendency and matched difference approaches for assessing interrater agreement.

Central tendency and matched difference approaches for assessing interrater agreement.

Differential Relationship of Person- and Task-Focused Leadership to Team Effectiveness: A Meta-Analysis of Moderators

Autonomy as a key resource for women in low gender egalitarian countries: A cross‐cultural examination

Contact Info

Product

Resources

About