Five Things You Should Know about Cost Overrun

Abstract

This paper gives an overview of good and bad practice for understanding and curbing cost overrun in large capital investment projects, with a critique of Love and Ahiaga-Dagbui (2018) as point of departure. Good practice entails: (a) Consistent definition and measurement of overrun; in contrast to mixing inconsistent baselines, price levels, etc. (b) Data collection that includes all valid and reliable data; as opposed to including idiosyncratically sampled data, data with removed outliers, non-valid data from consultancies, etc. (c) Recognition that cost overrun is systemically fat-tailed; in contrast to understanding overrun in terms of error and randomness. (d) Acknowledgment that the root cause of cost overrun is behavioral bias; in contrast to explanations in terms of scope changes, complexity, etc. (e) De-biasing cost estimates with reference class forecasting or similar methods based in behavioral science; as opposed to conventional methods of estimation, with their century-long track record of inaccuracy and systemic bias. Bad practice is characterized by violating at least one of these five points. Love and Ahiaga-Dagbui violate all five. In so doing, they produce an exceptionally useful and comprehensive catalog of the many pitfalls that exist, and must be avoided, for properly understanding and curbing cost overrun. Five Key Questions about Cost Overrun Cost overrun in large capital investment projects can be hugely damaging, incurring outsize losses on investors and tax payers, compromising chief executives and their organizations, and even leading to bankruptcy (Flyvbjerg et al. 2009, Flyvbjerg and Budzier 2011). Accordingly, cost overrun receives substantial attention in both the professional literature and popular media. Yet it is not always clear how cost overrun is defined, why it happens, and how to best 1 All authors have co-authored or authored publications based on the data, theories, and methods commented on by Love and Ahiaga-Dagbui (2018).

On de-bunking “Fake News” in the post-truth era: How to reduce statistical error in research

“…Lunn co-authored our first comment (Flyvbjerg et al, 2018) and would have been on the team for the present follow-up had he not died suddenly in February 2019. We missed him in preparing this note and will miss him going forward.…”

“…In a short forerunner to Flyvbjerg et al (2018), invited by the editor of Local Transport Today, 1 Flyvbjerg identified the above statement as a statistical error: https://doi.org/10.1016/j.tra.2019.06.011 ☆ We would like to dedicate this comment to the memory of our colleague Daniel Lunn, who was Professor of Statistics at the University of Oxford.…”

“…Other scholars would then be able to learn from the mistake and not repeat it, which is the way the academy is supposed to work. We further suggest that after having corrected the first error, the scholarly thing to do would have been for Love and Ahiaga-Dagbui to address each of the remaining 13 errors, like we addressed each of Love and Ahiaga-Dagbui's points of critique in Flyvbjerg et al (2018).…”

“…The editors of Transportation Research Part A recently received a comment by Peter Love, Lavagnon Ika and Dominic Ahiaga-Dagbui (2019) as a rejoinder to our critique (Flyvbjerg et al, 2018) of a paper by Love and Ahiaga-Dagbui (2018). However, as the rejoinder came in the form of a new paper, the editors kindly offered us the possibility of a re-rejoinder, which follows below.…”

Regression to the tail: Why the Olympics blow up

“…4 Extreme values, like those found here, are often erroneously seen as statistical outliers that are "a nuisance and must be removed" before analysis (Aguinis 2014: 3). Godfrey et al (2009) is an example of this approach, with more examples presented in Flyvbjerg et al (2018). But removal of extreme values is nearly always the wrong strategy.…”