Subjectivity and the Weighting of Performance Measures: Evidence from a Balanced Scorecard

Abstract

This study examines how different types of performance measures were weighted in a subjective balanced scorecard bonus plan adopted by a major financial services firm. Drawing upon economic and psychological studies on performance evaluation and compensation criteria, we develop hypotheses regarding the weights placed on different types of measures. We find that the subjectivity in the scorecard plan allowed superiors to reduce the “balance” in bonus awards by placing most of the weight on financial measures, to incorporate factors other than the scorecard measures in performance evaluations, to change evaluation criteria from quarter to quarter, to ignore measures that were predictive of future financial performance, and to weight measures that were not predictive of desired results. This evidence suggests that psychology-based explanations may be equally or more relevant than economicsbased explanations in explaining the firm's measurement practices. The high level of subjectivity in the balanced scorecard plan led many branch managers to complain about favoritism in bonus awards and uncertainty in the criteria being used to determine rewards. The system ultimately was abandoned in favor of a formulaic bonus plan based solely on revenues.

Multi-criteria analysis using latent class cluster ranking: An investigation into corporate resiliency

“…For example, the "Hawthorne Effect" (Ittner and Larcker, 1998a) is avoided by considering a sample of companies that had not implemented (to our knowledge) the balanced scorecard method. Ittner et al (2003;729) point to a "significant limitation" in many studies due to "the lack of data on nonfinancial or subjective performance dimensions, forcing researchers to use indirect proxies for the measures' informativeness". This study was able to use actual data on the 11 performance measures.…”

“…This ambiguity has created serious implementation problems. The subjectivity in assignment of the weights by evaluators, e.g., through AHP, has been identified as a major cause of failure of BSC and its eventual abandonment by a major financial services firm (Ittner et al, 2003). In the methodology we develop and employ here the weights are implicitly selected by the statistical model based on the similarity of the performance outcomes of different companies.…”

The Determinants of Salary and Bonus for Rank and File Employees

“…Moreover, Ittner et al (2003) …nd that the subjectivity of the performance evaluation increases over time if the evaluation process allows for su¢ cient discretion of the evaluator. In particular, they investigate the bonus awards in a large …nancial service …rm which uses the Balanced Scorecard system for performance measurement.…”

Incentives and managerial experience in multi-task teams: evidence from within a firm

“…There is also an earlier literature, for example, Ridgway (1956) criticises the idea of multidimensional incentives schemes. 3 Hoque and James (2000) survey 66 Austrailian manufacturing firms, Banker, Potter and Srinivsan (2000) look at 18 hotels, Malina and Selto (2001) consider multiple divisions of a large firm, Ittner, Larcker and Meyer (2003) conduct a case study in a large firm, Neely, Martinez and Kennerly (2004) consider 35 branches in one firm, Davis and Albright (2004) look at nine branches of a firm, Burgess et al (2004) evaluate a randomised trial incentive scheme in Job Centres in the UK. 4 Burgess et al (2004) are able to evaluate the introduction of a team based multi-task incentive scheme in Job Centres in the UK, where implementation was randomised.…”