Making biased but better predictions: The trade-offs strategists face when they learn and use heuristics

Abstract

The heuristics strategists use to make predictions about key decision variables are often learned from only a small sample of observations, which leads to a risk of inappropriate generalization when strategists misjudge regularities. Building on the statistical learning literature, we show how strategists can mitigate this risk. Strategies to learn heuristics that accept a bias, that is, a systematic deviation of predictions from actual outcomes, can outperform unbiased strategies because they can reduce the variance component of prediction error: the degree to which random fluctuations in observational data are inappropriately generalized. We demonstrate how strategists who are aware of the trade-off between bias and variance can learn heuristics more effectively if they are also aware of the relevant characteristics of their learning environment. We discuss the implications of our results for our understanding of heuristics, (dynamic) capabilities, and managerial cognitive capabilities, and we outline opportunities for empirical work.

Entrepreneurs as Scientists: A Pragmatist Approach to Producing Value Out of Uncertainty

“…The intuition is that entrepreneurs must decide how to modify and match their theories to problem spaces for which they do not have the full data, but a biased sample. Highly-revised theories may have a high fit with the biased sample, but come at the cost of poorly explaining the full problem as reflected in the full data (Ehrig & Schmidt, 2021b). Overfitted models are tuned to statistical noise and overly complex.…”

Supplementary schools with connections to religious organisations in the United Kingdom: A heuristic device for school leaders and researchers

“…A heuristic is helpful for as a device for discovery but is not ever intended to be a comprehensive description and that may be a matter of disappointment for those seeking such. There is also the compromise with the potential for greater bias within a heuristic device (Ehrig and Schmidt, 2019), which needs balancing with the potential for it to foster fast decision-making while being abandoned when it is no longer needed or superseded. This heuristic needs elaboration from experience but, as Bingham et al (2019) contend, a heuristic's early forms are more important for the structure they offer rather than their content.…”

“…Supplementary schools can be referred to as a type of school, but only to distinguish them from mainstream or private full-time schools, and there is no attempt here to claim a normative typology of schools as heuristics are meant to be incomplete and inherently inaccurate devices for discovery that can be constantly revised. They aim to bring attention to structuring decision-making (Bingham et al, 2019) and involve a compromise that while they may lead to better predications, they involve greater bias (Ehrig and Schmidt, 2019).…”

Theory‐based learning and experimentation: How strategists can systematically generate knowledge at the edge between the known and the unknown

“…Another avenue for further work is to link theory‐based learning with statistical learning. Validating or falsifying theories using statistics from an ex‐ante perspective is of course impossible: For some relations in a theory, data will only be available in the future, but theories can provide useful priors for statistical learning at a later stage (Ehrig & Foss, 2022; Ehrig & Schmidt, 2021; Griffiths & Tenenbaum, 2009). Moreover, strategists can employ a staged approach and use Bayesian learning, for example, to improve products in a test market (Zellweger & Zenger, 2021) or prototypes (Ehrig, Knudsen, & Rauh, 2022).…”