Multifaceted Conceptions of Self‐other Ratings Disagreement

Abstract

Self‐other ratings disagreement (S‐ORD) is the extent to which one's self‐rating of performance differs from ratings made by Other people (e.g., supervisors). As increasing number of organizations adopt self‐assessment as a tool for managing performance, S‐ORD is becoming a central concern in performance appraisal research. Most past research, however, only examines S‐ORD in terms of the levels of ratings, and ignores other forms of disagreement. This paper identifies 7 different forms of S‐ORD that are detectable through the use of confirmatory factor analysis (CFA). The causes for each form of S‐ORD are discussed, as well as its conceptual and practical implications. The procedure is illustrated by comparing 332 managers' self‐evaluations with their supervisors' evaluations. An extension of the CFA model to a full structural equation model is used to illustrate and discuss the fragmentary nature of existing approaches to S‐ORD. It is shown that valid inferences concerning S‐ORD cannot be made unless various forms of S‐ORD, and not merely the differences in overall ratings, are detected and dealt with.

A meta-analytic review of leadership impact research: Experimental and quasi-experimental studies

“…Along the same lines, we suggest that future research attempt to determine the extent to which data collected from single versus multiple sources may have affected the pattern of results reported from this study. Indeed, the effect of single source/method bias should be explored in future leadership intervention research particularly given the problems of interpreting common source data and differing levels of self-other agreement (Cheung, 1999;Dionne, Yammarino, Atwater & James, 2002). For example, findings reported by Lowe, Kroeck, and Sivasubramaniam (1996) showed substantially higher relationships for data collected from a single source with ratings of transformational leadership and performance outcome ratings.…”

Competence Development and Appraisal in Organizations

“…For the appraisal of competencies, there is a tendency towards the use of multi-rater or 360-degree appraisals of performance (especially manager performance) (see e.g., Cheung, 1999;Yammarino & Atwater, 1997). In 1996, almost all Fortune 500 firms were using 360-degree appraisal (see Yammarino & Atwater, 1997).…”

Three method factors explaining the low correlations between assessment center dimension ratings and scores on personality inventories

“…Self ratings showed a mean of M ¼ 3.5 versus an assessor rating mean of M ¼ 2.6. On all three dimensions, self ratings were significantly higher than assessor ratings (Analytical Skills, mean difference (Cheung, 1999) or self versus assessor ratings (Clapham, 1998).…”

“…Self ratings showed a mean of M ¼ 3.5 versus an assessor rating mean of M ¼ 2.6. On all three dimensions, self ratings were significantly higher than assessor ratings (Analytical Skills, mean difference (Cheung, 1999) or self versus assessor ratings (Clapham, 1998).The second part of the first hypothesis concerned AC other ratings (assessors) and NEO other ratings (peers). The mean correlation 1(b) was r ¼ 0.17, which is higher than the baseline of r ¼ 0.05.…”