Why Are Most Published Research Findings Under-Theorized?

Abstract

While the so-called replication crisis is increasingly discussed and addres sed through reformed research practices and institutional structures, this contribution diagnoses a theory or interpretation crisis and argues that the current emphasis on transparency, reproducibility, and reliability should be complemented by stronger efforts in terms of theory and validity. The article identifies different types of unsystematic (e.g., ad hoc, asymmetric, or trivial) theory building. Furthermore, objects of investigation, measure ments, and findings are not interpreted carefully in the light of sufficiently elaborate and well-justified theoretical concepts or frameworks. Different consequences of such shortcomings are discussed-whether unfruitful or implausible hypotheses are tested or the implications of findings are re main poorly understood or are not critically reflected. Readers are invited to engage not only with methodological literature and previous research, but also with theoretical works and particularly with literature on methods or strategies of theorizing, and to practice theory building based on a clearer understanding of the multiple meanings and functions of theory.

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

“…A series of articles in psychology have also pointed out a more fundamental problem contributing to insufficient replication and research waste [32][33][34][35][36][37][38][39]. To distinguish it from a 'replication crisis', it has been called a 'theory crisis' [34][35][36], wherein researchers frequently test vague and incorrect hypotheses/questions because the theory is often verbal (i.e., informal), so researchers can interpret it more freely (c.f., high researcher degrees of freedom [40]).…”

“…Even when written formally as mathematical models, theory is often poorly described, leading to misinterpretation by empiricists and a mismatch between theoretical predictions and empirical hypotheses. This has been called an 'interpretation crisis' [32] (see also, 'generalizability crisis' [38]). As the opening quote from John Tukey suggests, testing a precise hypothesis, derived from a wrong or misinterpreted theory, seems certainly a form of research waste or, at least, research inefficiency.…”

“…In psychology, several authors have recently pointed out that this statistical translation is problematic because many theories and hypotheses are verbal, so interpretations of the theories can become unconstrained or subjective and, therefore, ungeneralizable [32,[34][35][36][37][38]. In biology, such translation issues can certainly happen or even be prevalent based on our experience.…”

“…[64]; see also [65]). The third is hypothesis cherry-picking (termed by Krämer [32]), where researchers deliberately ignore alternative, theoretically valid hypotheses that do not support their initial beliefs or most favorable interpretation of their results.…”