Mandatory theorizing and synthesis: A comment on Phaf (2020)

Abstract

Should mandatory theorizing be a condition for researchers to publish articles? Should more synthesis be required, perhaps with the aid of meta-analysis? This short comment focuses on both questions that arise from Phaf’s article, “Publish less, read more” (2020).

Publish less, read more: Replies to Clegg, Wiggins, and Ostenson; and to Trafimow

“…A data-oriented focus (e.g., contenting oneself with a significant “effect”) further aggravates the isolation of experimental papers from other relevant publications. Trafimow (2020) is entirely correct that there are more than sufficient data for ad-hoc theorizing but that it does not happen, at least not enough. Due to the publication overload and the statistical illusion (see Phaf, 2020), experimental psychologists often perform and publish research while being ill-prepared (cf.…”

“…Ideally, one would compare these to theoretical estimates by formulating mathematical models that provide quantitative predictions and accuracy limits, but this does not seem very well possible in experimental psychology. Certainly, any attempt at quantifying these estimates from the data may suffer from the type of practical problems noted by Trafimow (2020). Both qualitative, narrative, reviews and quantitative meta-analyses, however, contribute to the type of meta-analytic thinking Cumming (2014) has argued for.…”

“…Although I very much appreciate Trafimow’s (2020) kind words, I am not sure whether the distinction between central and auxiliary hypotheses is all that important in an evolutionary conception of scientific development. I would argue that the distinction can only be made a posteriori when you know what the winning, and losing, hypotheses are.…”