Estimating the relationship between skill and overconfidence

Abstract

The Dunning-Kruger effect states that the low skilled are overconfident while the high skilled are more accurate in assessing their skill. In apparent support of this effect, many studies have shown that low performers overestimate their performance while high performers are more accurate. This empirical pattern, however, might be a statistical artifact caused by measurement error. We are the first paper to consistently estimate the Dunning-Kruger effect using an instrumental variable approach. In the context of exam grade predictions of economics students, we use students' grade point average as an instrument for their skill. Our results support the existence of the Dunning-Kruger effect. JEL: D03; I23PsycINFO classification: 2220; 3040; 3120 Keywords: Overconfidence, judgment error, measurement error, instrumental variable * We thank Christian Kerckhoffs and Alexander Vostroknutov for access to their courses. We further thank Adam Booij, Thomas Dohmen, Jonas Lang, Anna Zseleva, and seminar participants at Maastricht University and IZA and especially Nicolas Salamanca for valuable comments on earlier drafts of this paper.

Grade Expectations: Rationality and Overconfidence

“…Kruger and Dunning ( 1999 ) concluded that low-performing students significantly overestimate their performance, while high-performing students are more accurate in their forecasts. Feld et al ( 2017 ) pointed out that this effect could at least partially be explained by measurement errors. They showed that after correction for measurement errors (using instrumental variables) the Dunning-Kruger effect is still observed, but significantly weaker than before the correction.…”

“…Expectations move closer to the realized grade as students receive new information on their actual performance closer to the exam. Many other papers reject the hypothesis of rational expectations and confirm student grade overconfidence; see Kruger and Dunning ( 1999 ), Svanum and Bigatti ( 2006 ), Andrews et al ( 2007 ), Burns ( 2007 ), Jensen and Moore ( 2008 ), Khachikian et al ( 2011 ), Hossain and Tsigaris ( 2013 ), Feld et al ( 2017 ), Foster et al ( 2017 ), Serra and DeMarree ( 2016 ), and Sturges et al ( 2016 ).…”

Psychophysical deconstruction of the Dunning-Kruger effect

“…Since the seminal paper establishing the DKE, there has been debate over whether the effect derives from metacognitive differences between skilled and unskilled people (Kruger & Dunning, 1999), from general biases of self-estimation (Burson et al, 2006;Krueger & Mueller, 2002), or from statistical artefacts (Feld et al, 2017;Krajc & Ortmann, 2008;Krueger & Mueller, 2002). Despite vigorous defences of the metacognitive account (Dunning, 2011;Ehrlinger et al, 2008;Kruger & Dunning, 2002;Schlösser, Dunning, Johnson, & Kruger, 2013), and its enthusiastic dissemination through the wider culture, unambiguous evidence has not been provided, and the debate has persisted.…”

“…Competing accounts of the DKE focus on other mechanisms that might induce this negative correlation. One mechanism often invoked is regression to the mean (Ackerman, Beier, & Bowen, 2002;Burson, Larrick, & Klayman, 2006;Feld, Sauermann, & de Grip, 2017;Krueger & Mueller, 2002;Nuhfer, Cogan, Fleisher, Gaze, & Wirth, 2016;Nuhfer, Fleisher, Cogan, Wirth, & Gaze, 2017). A common method for studying the DKE involves ranking people by task performance and examining the relationship with estimation error, as indexed by the subtraction of actual from estimated performance; a negative correlation is virtually guaranteed in this scenario; just as, for any two imperfectly correlated random variables, x correlates negatively with y-x.…”

“…In concrete terms, assuming imperfect selfestimation, then any chance variations that increase errors will be inadequately tracked in self-estimation, pushing people down the ranks of performance and simultaneously biasing them towards over-estimation; and vice-versa for chance variations that reduce errors. This artefact is important to recognise, but relatively easy to eliminate, either by quantifying and controlling for unreliability in the measure of performance (Ehrlinger et al, 2008;Krueger & Mueller, 2002), or by using separate sub-sets of trials to index performance and to calculate estimation error (Burson et al, 2006;Feld et al, 2017;Klayman, Soll, González-Vallejo, & Barlas, 1999). If either of these steps is taken, the DKE is reduced in strength, but it is not eliminated, so further factors must also be at work.…”