Spillover Effects in Subjective Performance Evaluation: Bias and the Asymmetric Influence of Controllability

Abstract

We examine how subjective performance evaluations are influenced by the level and controllability of an accompanying measure of a separate performance dimension. In our experiment, supervisors evaluate the office administration performance of a hypothetical subordinate. We find that supervisors' subjective evaluations are directionally influenced by an accompanying objective measure of sales performance, even after excluding participants who perceive informativeness across measures. Consistent with concerns for fairness and motivation, we also find an asymmetric uncontrollability effect—supervisors' evaluations are higher when an uncontrollable factor decreases the subordinate's sales (i.e., they compensate for bad luck), but are not lower when the uncontrollable factor increases the subordinate's sales (i.e., they do not punish for good luck). This evidence suggests that supervisors use discretion provided to evaluate performance on one task to adjust for perceived deficiencies in the evaluation of performance on other tasks. Our study integrates theories of cognitive bias and motivation, highlighting the need to consider the potentially interactive effects of different performance measures in multi-task settings.

Promotion, Relative Performance Information, and the Peter Principle

“…In an extensive survey of 250 U.S. car dealerships, provide evidence that subjectivity is used to reduce workers' downside risk such as when workers face targets that are difficult to attain or when the firm is in a loss position. Similarly, Bol and Smith (2011) provide experimental evidence that evaluators use their discretion to adjust workers' performance ratings upwards when an uncontrollable factor decreases workers' performance, but make no adjustments when the same factor increases workers' performance. This result suggests that evaluators tend to compensate workers for bad luck, but do not punish workers for good luck.…”

Performance Measurement and Incentive Intensity

“…We think this category is worth considering when grouping performance measures. On the one hand, subjective measures are widely used by organizations because they help to solve the deficiencies in objective measurement, such as the presence of noise (see Bol and Smith, 2011 The literature on performance measurement has considered other classifications besides the one we employ in this study. Among the most frequently used we find the distinction between financial and non-financial measures, that between broad and narrow measures or between input and output measures (for a review of different classification schemes see, for example, Ittner and Larcker, 2002).…”

“…More precisely, one of the main reasons for using subjective measures is the risk associated with objective indicators (Baker, 1994). This idea is also tested by Bol and Smith (2011). By means of an experiment, these authors show that supervisors use subjective measures to compensate for the deficiencies in objective measurement and, in particular, to offset the negative effects caused by uncontrollable factors.…”

Halo Effect in Subjective Performance Evaluation Bias

“…However, objective performance evaluation is defined as judgment based on personal impressions that is unquantified such as innovation, creativity, loyalty, the ability to work together, sharing knowledge, leadership, or the ability to communicate (Baker et al 1994;Bella Vance et al 2013, Bol 2008). Both models are interplay one another (Bol and Smith 2011), can be adopted by the company simultaneously (Prendergast 1999), and actually not mutually exclusive but rather complementary (Breuer et al 2013;Bommer et al 1995;Prendergast and Topel 1996).…”