What makes a good theory, and how do we make a theory good?

Abstract

I present an ontology of criteria for evaluating theory to answer the titular question from the perspective of a scientist practitioner. Set inside a formal account of our adjudication over theories, a metatheoretical calculus, this ontology comprises: a) Metaphysical commitment, the need to highlight what parts of theory are not under investigation, but are assumed, asserted, or essential. b) Discursive survival, the ability to be understood by interested non-bad actors, to withstand scrutiny within the intended (sub)field(s), and to negotiate the dialectical landscape thereof. c) Empirical interface, the potential to explicate the relationship between theory and observation, i.e., how observations relate to, and affect, theory and vice versa. d) Minimizing harm, the reckoning with how theory is forged in a fire of historical, if not ongoing, abuses — from past crimes against humanity, to current exploitation, turbocharged or hyped by machine learning, to historical and present internal academic marginalization. This work hopes to serve as a possible beginning for scientists who want to examine the properties and characteristics of theories, to propose additional virtues and vices, and to engage in further dialogue. Finally, I appeal to practitioners to iterate frequently over such criteria, by building and sharing the metatheoretical calculi used to adjudicate over theories.

Reclaiming AI as a theoretical tool for cognitive science

“…The way to make progress is through the meticulous development of theoretical ideas, informed by formal and computational modelling, drawing out limitations, consequences, and building solid knowledge along the way. In other words, what we advocate is more theoretical thinking (see also Guest, 2023;Guest & Martin, 2021;van Rooij & Baggio, 2020), and less (unthinking) machine learning or less confusion between machine learning and theory (cf. Andrews, 2023).…”