Rating Level and Accuracy as a Function of Rater Personality

Abstract

We investigated the predictability of rating level and two measures of rating accuracy from rater Agreeableness (A) and Conscientiousness (C) scores of the Five Factor Model. One hundred and twenty-six students made peer ratings after participating in several group exercises under conditions designed to emulate the modal peer rating system in which raters had low accountability for their ratings. Scores were correlated with average rating level (r ¼.18, po.05) and both measures of rating accuracy (po.05) and C scores were correlated with rating level (À.20, po.05) and both measures of rating accuracy (po.05). As suggested by Bernardin, Villanova, and Cooke (Journal of Applied Psychology, 85, 232-236, 2000), raters who were both more agreeable and less conscientious made the most lenient and least accurate ratings. Contrary to Yun, Donahue, Dudley, and McFarland (International Journal of Selection and Assessment, 13, 97-107, 2005), more agreeable raters also tended to rate the least effective performers more leniently than did other raters.

The impact of rater agreeableness and rating context on the evaluation of poor performance

“…High-agreeableness individuals have shown a particular propensity to be motivated to give lenient ratings (Bernardin, Cooke, Ross & Villanova, 2000;Jawahar, 2001) perhaps because they value conflict-free, harmonious relationships with others and have a willingness to compromise their own interests for the sake of others (Goldberg, 1992). When combined with low rater conscientiousness, high agreeableness has been found to result in particularly lenient ratings (Bernardin, Tyler, & Villanova, 2009). …”

“…It has been shown that raters are not always willing to rate accurately: Managers can make accurate private judgments of employee performance but that these may not be the same as the ratings they give in public (Murphy & Cleveland, 1995;Tziner, Murphy, & Cleveland, 2005). This could cause particularly serious problems when poor work performance needs to be identified and discussed (Arvey & Murphy, 1998;Bernardin, Tyler, & Villanova, 2009;Heslin, Latham, & VandeWalle, 2005).…”

“…However, the effects of the interactions between rating context and rater disposition on rating behaviour remain unclear. There has also been some debate about whether leniency is caused by rater disposition when very poor performance is rated (Yun, Donahue, Dudley, & McFarland, 2005;Bernardin, Tyler, & Villanova, 2009). In this study we used an experimental design within a working population to examine how rating context and rater agreeableness combined to influence PA ratings of poor performance.…”

Managing the Interpersonal Aspect of Performance Management

“…This measure could be examined as a state measure (influenced by the ratee and rating environment), a trait measure (a relatively stable attribute of the rater), or both. With regard to traits, Bernardin et al (2009) raters who were high on agreeableness and low on conscientiousness were also especially likely to be lenient in their PA. Other rater traits and rater states could be investigated in the context of an organizational environment that elicits or constrains trait-relevant behaviors (see the trait activation theory of Tett & Burnett, 2003).…”

“…Such interventions might have main effects across all raters or might interact with individual differences in rater characteristics. For example, absent any formal intervention, conscientious raters generally tend to provide less lenient and more accurate ratings (e.g., Bernardin, Tyler, & Villanova, 2009), and in some cases, these raters seem to be less influenced by contextual demands that are consistent with conscientiousness (e.g., accountability to audience; Roch, Ayman, Newhouse, & Harris, 2005). Conversely, these interventions might be most effective in those raters who are especially susceptible to rating leniency.…”