Top performers are not the most impressive when extreme performance indicates unreliability

Abstract

The relationship between performance and ability is a central concern in the social sciences: Are the most successful much more able than others, and are failures unskilled? Prior research has shown that noise and self-reinforcing dynamics make performance unpredictable and lead to a weak association between ability and performance. Here we show that the same mechanisms that generate unpredictability imply that extreme performances can be relatively uninformative about ability. As a result, the highest performers may not have the highest expected ability and should not be imitated or praised. We show that whether higher performance indicates higher ability depends on whether extreme performance could be achieved by skill or requires luck.regression to the mean | randomness | social learning | performance evaluation | ecological rationality E xtreme performance attracts people's attention. People tend to believe the most successful are the most skillful and that failures lack skill (1, 2). A tendency to imitate the most successful has also been argued to be a basic universal trait that is shaped by evolution and promotes adaptiveness (3, 4). However, is success necessarily an indication of skill and worthy of praise and imitation and failure an indication of lack of skill?Clearly, observed performance is not always a reliable indicator of skill. Chance events outside the control of individuals often influence performance (5-7). Moreover, such chance events rarely average out over time. Instead, due to "rich-get-richer" dynamics and "Matthew effects" (8), success usually breeds success and failure breeds failure. For example, individuals with early success might be given more resources and instruction, or consumers may favor products with a high market share (9, 10). Prior research has shown how such processes can amplify chance events and produce a weak association between performance and ability (11-13), leading to a distribution of outcomes that is both unpredictable and highly unequal (14). In such settings, extreme success and failure are, at best, only weak signals of skill. The highest performers may be more able than others and the lowest performers less able than others, but one should not expect their skill level to be very far from the mean (15).These prior contributions show that performance and skill may be weakly associated due to noise and rich-get-richer dynamics, but they do not challenge the idea that higher performers are likely more skilled and worthy of imitation. Even if the highest performers are only marginally more skilled than others, it makes sense to imitate them. In this paper, we show that noise and richget-richer dynamics can have more counterintuitive implications that go beyond the conventional understanding of regression to the mean. Noise and rich-get-richer dynamics not only introduce unpredictability but also change how much one can learn from extreme performances and whether higher performance indicates higher skill. In particular, we show that when noise and rich-getricher dy...

Good Night, and Good Luck: Perspectives on Luck in Management Scholarship

“…Each perspective is closely tied to a particular literature that entails as yet unsolved puzzles that suggest promising directions for future research. For example, while prior studies in psychology uniformly suggest that people tend to mistake luck for skill when evaluating performance differences (Baron & Hershey, 1988;Kahneman, Slovic, & Tversky, 1982;Langer, 1975;Nisbett & Ross, 1980;Rosenzweig, 2007;Ross & Nisbett, 1991), recent studies point at an asymmetry in the evaluations between successes and failures (Denrell & Liu, 2012;Pizarro, Uhlmann, & Salovey, 2003;Uhlmann, Pizarro, & Diermeier, 2015). While people appreciate the role of (bad) luck in failures, they do not necessarily do the same when it comes to explaining success.…”

“…But prior studies have not challenged the idea that top performers are the most skilled and worthy of reward and imitation. More recent studies show that the belief that the top performers are the most capable is flawed because exceptional success usually occurs in exceptional circumstance (Denrell & Fang, 2010;Denrell et al, 2013;Denrell & Liu, 2012). Top performers have been shown to be lucky for having benefitted from rich-get-richer dynamics that boosted their initial fortune.…”

“…Still others argue that while good and bad luck can happen to anyone, some are more prepared than others (de Rond, 2014;Dew, 2009), for example, by being mindful enough to rebound from bad luck (Weick & Sutcliffe, 2006), or by securing a higher 'return on luck' (Collins & Hansen, 2011). Some even argue that others' systematic underestimation of luck can signal profitable opportunities (Denrell & Liu, 2012;Lewis, 2003;Mauboussin, 2012;Powell, Lovallo, & Fox, 2011;Taleb, 2001). …”

“…Can a similar unifying formal framework be developed to explain regularities in performance persistence, career development, firm size, risk taking, and survival? Chance models exist for each of these regularities (e.g., Denrell, 2004;Denrell, 2008;Denrell & Liu, 2012;Denrell & Shapira, 2009;Geroski, 2005;Levinthal, 1991;March & March, 1977) but these separate models have not yet been integrated theoretically. Can a formal framework be developed from which all of these regularities could be derived?…”

Are high-growth firms overrepresented in high-tech industries?

“…While "a propensity for innovation emerges in general as a firm's growth driver" according to Vivarelli (2013), others stress that the self-reinforcing dynamics in the economy may lead to a relatively weak association between the ability to innovate and actual performance, and even if firms are successful in innovation and benefit from it, it is not clear that they will grow (Kirchoff 1994;Geroski et al 1997;Coad and Hölzl 2010;Denrell and Liu 2012;Coad et al 2014b:8).…”