Commentary—Relevancy Is Robust Prediction, Not Alleged Realism

Abstract

Remember James Boswell, ninth Laird of Auchinleck, author of the famous maxim that the road to hell is paved with good intentions? Trying to build realistic theories differs dramatically from having correct explanatory theories tested on objective criteria, e.g., verifiable prediction. Evaluating theories on whether assumptions are realistic is potentially subjective, biased, and arbitrary. A theory's domain determines whether its assumptions are sufficiently realistic and when assumptions must hold and to what degree, so testing assumptions in isolation puts an unnecessary burden on the assumptions (i.e., they must hold everywhere). For theories explaining cooperation and information exchange, predictions reveal that the prisoner's dilemma assumptions (only two prisoners, four possible outcomes, two possible actions, etc.) are sufficiently realistic. For theories explaining prisoner sentencing guidelines and probation policy, predictions might suggest otherwise. Scientific methods allow the evaluation theories on criteria such predictive accuracy, reliability, validity, and robustness-not realism. When multiple explanatory theories survive initial testing, one derives conflicting predictions. Tragic consequences can occur when theory adoption depends on whether assumptions are disliked, unpopular, or exclude a favorite variable. Pity journals that reject models with insightful new implications because the assumptions are too simple or merely disliked. The term "unrealistic" sometimes means personally disliked.

Online Display Advertising: The Influence of Web Site Type on Advertising Effectiveness

“…Thus in the next Study 2, we conducted a more internally valid test of our hypotheses by using a single target brand instead of multiple brands, and a fictional brand without any associations instead of real brands. This approach is consistent with guidelines for theory testing, which recommend the use of methods to maximize internal validity of test procedures (Calder & Tybout, ; Shugan, ).…”

Rethinking some of the rethinking of partial least squares

“…In his seminal treatise on the philosophy of science, Conjectures and Refutations , Popper (1962) posited that prediction is the primary criterion for evaluating falsifiability, and that all explanatory theories must “rise and fall based on their objective predictions” (Shugan, 2009, p. 994). Similarly, Steenkamp and Baumgartner (2000, p. 197) note that:…”

Estimating aggregate consumer preferences from online product reviews

“…All in all, the available hit rates are in accordance with common standards (see, e.g., Moore (2004) for corresponding values in different conjoint validation studies). Although the ability to predict the dependent variable in a satisfactory way is not sufficient to substantiate the overall adequacy of the suggested NBR approach, this at least provides a certain level of confidence (Shugan, 2009). …”