The Control of Bias in Ratings: A Theory of Rating

Abstract

Based on several years of research and a careful analysis of the rating process Wherry developed a theory of rating. An accurate rating is seen as being a function of three major components: Performance of the ratee, observation of that performance by the rater, and the recall of those observations by the rater. Cast in a mold of classical psychometric theory each of these components is seen as consisting of a systematic portion and a random portion. The systematic portion of each component is further broken down. The performance of the ratee is a combination of true ability or aptitude for the job and the influence of the environment. What the rater observes is a function the performance of the ratee and bias of observation and what the rater recalls is a result of those observations combined with a bias of recall. The development of the theory of rating unfolds by defining the various factors that affect each of these components in a series of linear equations. Various theorems and corollaries are proposed which should lead to a maximization of the true ability component of the ratee and minimize environmental influence and the bias and error components. The theorems and corollaries suggest testable hypotheses for the researcher in performance evaluation.

Getting Rid of Performance Ratings: Genius or Folly? A Debate

“…There is a long history of research on performance rating and performance appraisal (for reviews, see Bernardin & Beatty, 1984;DeCotiis & Petit, 1978;DeNisi, 2006;DeNisi, Cafferty, & Meglino, 1984;Ilgen & Feldman, 1983;Landy & Farr, 1983;Milkovich & Wigdor, 1991;Murphy & Cleveland, 1991Wherry & Bartlett, 1982), and although different reviews highlight different strengths and weakness of the methods that are used in organizations to measure job performance via rating scales or other similar measures, it is fair to say that none of these reviews leads to the conclusion that performance rating is particularly successful either as a tool for accurately measuring employee performance or as a component of a broader program of performance management. Nearly a century of research on performance appraisal (see, for example, Austin & Villanova, 1992) suggests that there is a longstanding history of problems with performance rating and little reason to believe that these problems will be solved in the foreseeable future.…”

Applying the social relations model to self and peer evaluations

“…Readers interested in obtaining a copy of the SOREMO program may access the program at http://nw3.nai.net/~dakenny/kenny.htm Applying the social relations model 509 generally work closely together. This proximity increases peers' opportunities to observe performance-related behaviors which presumably increases the accuracy of their ratings (Wherry and Bartlett, 1982). Second, peers probably observe a less biased sample of behavior than other rater sources.…”

Brokering Trust to Enhance Leadership: A Self-Monitoring Approach to Leadership Emergence

“…Our threeitem measure of job performance was based on confidential ratings provided by each individual's direct supervisor. Performance ratings that are collected for research purposes tend to more reliable and valid than those obtained for administrative purposes (Wherry & Bartlett, 1982). The three items that made up our measure were selected on the basis of multiple discussions with a panel comprising the firm's human resources director, a long tenured member with broad knowledge of the company, and four employees who represented a range of different job types at the firm.…”