Prediction and explanation in social systems

Abstract

Historically, social scientists have sought out explanations of human and social phenomena that provide interpretable causal mechanisms, while often ignoring their predictive accuracy. We argue that the increasingly computational nature of social science is beginning to reverse this traditional bias against prediction; however, it has also highlighted three important issues that require resolution. First, current practices for evaluating predictions must be better standardized. Second, theoretical limits to predictive accuracy in complex social systems must be better characterized, thereby setting expectations for what can be predicted or explained. Third, predictive accuracy and interpretability must be recognized as complements, not substitutes, when evaluating explanations. Resolving these three issues will lead to better, more replicable, and more useful social science.

The power of noise and the art of prediction

“…This implies that, if model A is better than model B given infinite data, due to bias-variance trade-off, there is no guarantee that the former will be better than the latter given finite data or a particular dataset. Therefore, we also need "predictive modeling" (Donoho, 2015;Hofman et al, 2017), which is generally agnostic about a data generating mechanism and allows multiple models to learn from and work on multiple datasets. Some of these are used to train the models, others are put aside as test sets, just as we turn a ball many times and each time we make a prediction about the patterns on the side we do not see using the information on the side we can see.…”

“…Some of these are used to train the models, others are put aside as test sets, just as we turn a ball many times and each time we make a prediction about the patterns on the side we do not see using the information on the side we can see. The performances of the trained models are then judged against a common task, usually, predictive accuracy on test sets, which is easy-to-understand and can be compared across datasets and over time (Breiman, 2001;Donoho, 2015;Hofman et al, 2017;James et al, 2015).…”

Interdependence and Superordinate Goals: The Revenge of the Dualists

“…Not resolving these "bewildering complexities" has left researchers in the social (e.g., economic, humanistic, philosophic, networks, game theory) disciplines struggling to predict the outcomes of basic interactions, exemplified by the difficulty in replicating experiments (Nosek 2015); left them aimless (Hofman et al 2017); and left them stunned by the achievements of their colleagues in the hard sciences (e.g., physics, chemistry, biology, engineering). The philosophy or history of science, where the search for truth goes to die, has replaced the foundations inherent in science with endless debate (Nickels 2017).…”

The empiricist’s challenge: Asking meaningful questions in political science in the age of big data

“…Here, we find a strong prominence of studies concentrating on statistical predictions of various political outcomes based on signals found in digital trace data (see Hofman, Sharma, & Watts, 2017;Schoen et al, 2013). In style and design, these studies follow computer science papers attempting to predict social phenomena or economic outcomes based on digital trace data (e.g., Choi & Varian, 2012).…”