A Longitudinal Examination of Rater and Ratee Effects in Performance Ratings

Abstract

The consistency and loci of leniency, halo, and range restriction effects in performance ratings were investigated in a longitudinal study. Ratings were provided by approximately 90 supervisors in a metropolitan police department, who rated approximately 350 police-rank subordinates on five occasions over a three and onehalf year period. Rating effects were computed separately as raterand ratee-based statistics, and intercorrelated among the five rating periods. T h e nature of the data set made it possible to hold either raters or ratees constant for each analysis, thus permitting inferences regarding the sources of reliable variance in effects as due to raters or ratees. It was concluded that reliable variance in mean ratings is partly attributable to ratees, but mainly introduced by raters. Reliable halo variance is attributable to raters, and range restriction is a product of stable group performance variability within intact ratee groups. Implications of these results for future rating process research are discussed.THE observation that performance rating research occupies a prominent position in the industrial psychology literature is becom-Reprint requests should be sent to Robert J. Vance,

Implications of Three Causal Models for the Measurement of Halo Error

“…It does not distinguish among the General Impression, Salient Dimension, and Inadequate Discrimination models of halo error with respect to the type of rater error that may have occurred. The distinction between rater errors and rating errors is not new (e.g., Borman, 1975;Feldman, 1986;Vance, Winne, & Wright, 1983), but it seems to have been ignored in much of the recent literature. The developments presented here emphasize 428 the importance of maintaining this distinction, as well as suggesting that an appropriate correlational measure of halo rating error can be obtained irrespective of which of the three types of halo rater error may be operating.…”

Self–other rating agreement in leadership: A review

“…However, this congruence among ratings, as shown by the descriptive data, may be the result of subordinates changing their ratings and not the result of managers becoming more self-aware and thus revising their self-ratings to fit feedback. Vance, Winne, and Wright (1983) examined performance ratings provided by 90 supervisors who rated 350 subordinates on five occasions over three and one-half years. The nature of their data set allowed them to hold either raters or ratees constant, thereby enabling inferences regarding the sources of reliable variance due to raters or ratees.…”

Getting Rid of Performance Ratings: Genius or Folly? A Debate

“…Simplifying the cognitive demands and reducing, through simplification of the process and forms, the motivational barriers for managers is also likely to be helpful (Efron & Ort, 2010). Finally, performance review panels might be used, which would have similar advantages and disadvantages as promotion panels (Catano et al, 2007;Church, 1995;Gilliland & Langdon, 1998;Kozlowski, Chao, & Morrison, 1998;Vance, Winne, & Wright, 1983;Werner & Bolino, 1997). In our view, it is the consequences of leaders assigning and communicating evaluative judgments expressed as ratings that matter the most, not the ratings themselves.…”