When Rating Format Induces Different Rating Processes: The Effects of Descriptive and Evaluative Rating Modes on Discriminability and Accuracy

Abstract

Purpose-We examined how different kinds of rating formats, and their interaction with purposes of rating (administrative vs. developmental), induced different performance rating processes and their consequences for rating accuracy. Design/methodology/approach-In two experiments, participants rated seven targets presented via videotapes using modes of rating giving access to: a) descriptive knowledge (rating scales were a target's observable behaviors: Descriptive Behavior-DB), b) evaluative knowledge (rating scales were others' behaviors that the target tended to afford: Evaluative Behavior-EB), or c) a mix of the two knowledge types (rating scales were traits). Indexes of discriminability (within-and between-ratee discriminability) and of accuracy (differential elevation and differential accuracy) were collected. Findings-The results showed that EB rating scales led to higher between-ratee discriminability and differential elevation than other modes of rating, whereas DB rating scales led to higher within-ratee discriminability than the other modes. Implications-Our results indicate that EB rating scales are more suited to comparing different ratees (e.g., an administrative purpose for rating), whereas DB scales are more suited to identifying strengths and weaknesses of a particular ratee (e.g., a developmental purpose). Originality/value-Our experiments are the first to apply dual-knowledge (descriptive vs. evaluative) theory to a performance appraisal context and to examine rating purpose in interaction with these two forms of person knowledge. The results, consistent with theoretical predictions, indicate that using rating scales with different types of content as a function of the rating purpose will produce more appropriate performance ratings.

Performance Management: A Scoping Review of the Literature and an Agenda for Future Research

“…There remains considerable diversity in this theme. While there is some interest among researchers concerning the merits of traditional format issues linked to behavioral measures of performance (three articles), graphic ratings (Yun, Donahue, Dudley, & Mcfarland, 2005), and behavior versus trait ratings (Cambon & Steiner, 2015), we also observed research designed to understand some of the newer PM formats such as competency-based evaluation (six articles, for example, Catano, Darr, & Campbell, 2007; Cheng, Dainty, & Moore, 2005; Molleman & der Vegt, 2007) and the balanced scorecard (Chan, 2006). Although less prevalent, we saw the use of narratives as opposed to purely numbers in two articles (Brutus, 2010).…”

Business strategy, performance appraisal and organizational results

“…The administrative category establishes that firms may use PA to mainly administer those decisions related to compensation issues, such as salary increases. Other type of administrative decisions that can also be supported by this PA category are the necessity of adoption of disciplinary actions, or decisions related to the convenience of workers' retention (Cambon and Steiner, 2015). Therefore, the administrative PA results would be mostly used to stablish comparisons among workers at the firm (Cleveland et al, 1989).…”

Using the shifting standards model of stereotype‐based judgments to examine the impact of race on compensation decisions

“…For example, the shift in standards would apply to other stereotype‐relevant characteristics, influencing judgments of competence and expected performance (e.g., Biernat & Fuegen, 2001; Lynch & Finkelstein, 2015) and impacting compensation and hiring/promotion decisions. There are additional implications for performance appraisals and hiring decisions for employees, such as differences in formal versus informal evaluation practices (Biernat et al., 2010) and rating format designs (e.g., descriptive and evaluative ratings; Cambon & Steiner, 2015).…”