Within-group agreement: On the use (and misuse) of rWG and rWG(J) in leadership research and some best practice guidelines

Biemann, Torsten; Cole, Michael S.; Voelpel, Sven C.

doi:10.1016/j.leaqua.2011.11.006

Cited by 269 publications

(290 citation statements)

References 67 publications

Supporting

Mentioning

275

Contrasting

Unclassified

Order By: Relevance

“…The F-ratios were all highly significant, and the ICC(1) values were far above the minimum effect size of .05 (Biemann et al, 2012;LeBreton & Senter, 2008). This indicated that classes differed significantly in teaching presence, social presence, and cognitive presence.…”

Section: Data Aggregationmentioning

confidence: 89%

“…The intra-class correlation 2 (ICC(2)) was calculated for teaching presence, social presence, and cognitive presence to measure within-class agreement, and intra-class correlation 1 (ICC(1)) was calculated as an index of betweenclass differences (Biemann, Cole, & Voelpel, 2012;Conway & Briner, 2012;Glisson & James, 2002;LeBreton & Senter, 2008;Lüdtke et al, 2006;Zohar & Luria, 2010).…”

Section: Data Aggregationmentioning

confidence: 99%

See 1 more Smart Citation

The teacher as leader: Effect of teaching behaviors on class community and agreement

Rubin

Fernandes

2013

IRRODL

View full text Add to dashboard Cite

This article examines the effects of teaching behaviors in online university classes, focusing on the agreement among class members. Literature on group leaders' effects on group agreement about workplace climate is reviewed. Hypotheses are generated about the effects that teachers of online courses, as class leaders, have on both the level and agreement about the community of inquiry. They are tested with a sample of 874 students in 126 online courses. The aggregate class level and strength of agreement about the teaching presence have significant effects on the level and agreement about cognitive presence and social presence. Although the aggregate levels and agreement about community of inquiry are related, different patterns emerge.The paper explores the interaction effects of level and agreement, finding that in classes with high levels of teaching presence, the higher the agreement about teaching presence, the higher the agreement about cognitive and social presence especially for classes reporting stronger levels of cognitive and social presence. In classes with lower levels of teaching presence, agreement has a different effect.

show abstract

Section: Data Aggregationmentioning

confidence: 89%

Section: Data Aggregationmentioning

confidence: 99%

The teacher as leader: Effect of teaching behaviors on class community and agreement

Rubin

Fernandes

2013

IRRODL

View full text Add to dashboard Cite

show abstract

“…To ensure that ratings could be aggregated, we evaluated inter-rater agreement (IRA) following literature recommendations [61,62], using three families of indices: James et al's r WG(J) [63,64] (based on multiple null distributions) [65], Brown and Hauenstein's a WG(J) [66]; and the adjusted average deviation index A DMJ(adj) [67]. In addition to the arithmetic mean of each uMARS score, we calculated a "response databased weighted mean" (WDMEAN) [68].…”

Section: Data Analysesmentioning

confidence: 99%

Assessing the Quality of Mobile Phone Apps for Weight Management: User-Centered Study With Employees From a Lebanese University (Preprint)

Bardus¹,

Ali²,

Demachkieh³

et al. 2018

Preprint

View full text Add to dashboard Cite

Background: Evaluating the quality of mHealth apps for weight loss and weight management is important to understand whether these can be used for obesity prevention and treatment. Recent reviews call for more research on multi-dimensional aspects of app quality, especially involving end-users, as there are already many expert reviews on this domain. However, no quantitative study has investigated how lay persons see popular apps for weight management and how they perceive different dimensions of app quality. Objective: To explore how lay persons evaluate the quality of six free weight management apps (MyDietCoach, SparkPeople, Lark, MyFitnessPal, MyPlate, MyDietDiary), which achieved the highest quality ratings in a related and recent expert review. Methods: A user-centered study was conducted with 36 employees of a Lebanese university. Participants enrolled in the study on a rolling basis between October 2016 and March 2017. Participants were randomly assigned an app to use for two weeks. App quality was evaluated at the end of the trial period using the Mobile App Rating Scale (user version, uMARS). uMARS assesses the dimensions of 'engagement', 'functionality', 'aesthetics', 'information', and 'subjective quality' on 5-point scales. Internal consistency and inter-rater agreement were examined. The associations between uMARS scores and background characteristics were also explored using non-parametric tests. Analyses were completed in November 2017. Results: Overall, the six apps were of moderately good quality (Md uMARS score=3.6, IQR=0.3). The highest total uMARS scores were achieved by Lark (M=4.0, SD=0.5) and MyPlate (M=3.8, SD=0.4), which also achieved the highest subjective quality scores (Lark: M=3.3, SD=1.4; MyPlate: M=3.3, SD=0.8). 'Functionality' was the highest-rating domain (Md=3.9, IQR=0.3), followed by 'aesthetics' (Md=3.7, IQR=0.5), 'information' (Md=3.7, IQR=0.1), and 'engagement ' (Md=3.3, IQR=0.2). 'Subjective quality' was judged low (Md=2.5, IQR=0.9). Overall, 'subjective quality' was strongly and positively related (p<0.001) with total uMARS score (rho=0.75), 'engagement' (rho=0.68), 'information' and 'aesthetics' (rho=0.60), but not 'functionality' (rho=0.40, p=0.02). Higher 'engagement' scores were reported among healthy (p=.003) and obese individuals (p=.034), who also showed higher total uMARS (p=.038), and 'subjective quality' (p=.047) scores. Conclusion: Even if the apps were considered highly functional, they were relatively weak in engagement and subjective quality scores, which indicate a low propensity of using the apps in the future. As engagement was the sub-domain most strongly associated with subjective quality, app developers and researchers should focus on creating apps that are engaging, holding constant the functionality, aesthetics, and information quality. The tested apps (in particular Lark and MyPlate) were perceived as more engaging and of higher quality among healthy, obese individuals, making them a promising mode of delivery for self-directed interventions...

show abstract

“…This index refers to the irregular results of distributional properties of one's sample and provide accurate and interpretable estimates of rater agreement which allow researcher to control the response biases such as central tendency and social desirability (Biemann et al, 2012). r wg is a measure of inter rater agreement as unifying factor and procedure to justify aggregating individual perceptions to the proposed level of analysis (Biemann et al, 2012).…”

Section: The R Wg Indexmentioning

confidence: 99%

“…Biemann et al (2012) have suggested that within-group agreement (r wg values) above the conventional acceptable value of 0.7 can be sufficient to support the aggregation to the group level for variables. Since the agreement is tested for each shared property measure of each unit, the high values of r wg for intended variables provide empirical support for inter-rater agreement within the unit.…”

Section: The R Wg Indexmentioning

confidence: 99%

Multilevel Approach In Organizational Research: A Review Of Dual-Level Transformational Leadership

Pourbarkhordari

Zhou

Pourkarimi

2016

ESJ

View full text Add to dashboard Cite

Due to the nature of organizations as hierarchically nested systems, multilevel theoretical models are relevant to the vast majority of organizational phenomena; hence, the main purpose of this review was clarifying the importance of conceptualization and testing theories at multiple levels of analysis. By doing this, the current study added to the gloom by providing a comprehension literature review with explicit consideration of multilevel directed theory and measures in organizational studies. Specifically, this review answered call for deliberate distinction between individual and group levels of analysis in leadership research, in particular transformational leadership (TFL) which in recent years, based on well-grounded theory stating to multiple levels is scarce. Our multilevel approach in examining organizational phenomena at different levels corroborates the recommendation to investigate the impact of constructs and their relationships at multiple levels of analysis. Finally, two prominent measurement models which have been conducted to exhibit dual level transformational leadership behaviors were recommended.

show abstract

Within-group agreement: On the use (and misuse) of rWG and rWG(J) in leadership research and some best practice guidelines

Cited by 269 publications

References 67 publications

The teacher as leader: Effect of teaching behaviors on class community and agreement

The teacher as leader: Effect of teaching behaviors on class community and agreement

Assessing the Quality of Mobile Phone Apps for Weight Management: User-Centered Study With Employees From a Lebanese University (Preprint)

Multilevel Approach In Organizational Research: A Review Of Dual-Level Transformational Leadership

Contact Info

Product

Resources

About