Preregistration in diverse contexts: a preregistration template for the application of cognitive models

Abstract

In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to increase rigour and transparency in science as a potential response to the challenges posed by the replication crisis. Many of these reforms—including the increasingly used preregistration —have been designed for purely experimental work that tests straightforward hypotheses with standard inferential statistical analyses, such as assessing whether an experimental manipulation has an effect on a variable of interest. But psychology is a diverse field of research. The somewhat narrow focus of the prevalent discussions surrounding and templates for preregistration has led to debates on how appropriate these reforms are for areas of research with more diverse hypotheses and more intricate methods of analysis, such as cognitive modelling research within mathematical psychology. Our article attempts to bridge the gap between open science and mathematical psychology, focusing on the type of cognitive modelling that Crüwell et al. (Crüwell S, Stefan AM, Evans NJ. 2019 Robust standards in cognitive science. Comput. Brain Behav. 2 , 255–265) labelled model application , where researchers apply a cognitive model as a measurement tool to test hypotheses about parameters of the cognitive model. Specifically, we (i) discuss several potential researcher degrees of freedom within model application, (ii) provide the first preregistration template for model application and (iii) provide an example of a preregistered model application using our preregistration template. More broadly, we hope that our discussions and concrete proposals constructively advance the mostly abstract current debate surrounding preregistration in cognitive modelling, and provide a guide for how preregistration templates may be developed in other diverse or intricate research contexts.

Preregistration: Definition, Advantages, Disadvantages, and How It Can Help Against Questionable Research Practices

“…Above, we described common characteristics of current preregistration templates (e.g., Crüwell & Evans, 2019;Kirtley, Lafit, Achterhof, Hiekkaranta, & Germeys, 2020;Krypotos et al, 2019;Van den Akker et al, 2019;Veer & Giner-Sorolla, 2016). As mentioned previously, however, different types of studies need different preregistration elements, and more templates are being introduced 4.0.6 Remaining sections..…”

The Formal Model article format: justifying modelling intent and a critical review of data foundations through publication

“…Without peer review, there is no chance to identify problems with experimental design. Registered reports with a journal allow these plans to be peer-reviewed and improved through the process (Crüwell and Evans 2021, Montoya et al 2021, Wiseman et al 2019. It can also be used to encourage the publication of negative results as the journal in which the report was reviewed agrees in principle to publish the results (Parker et al 2019).…”

Neurocomputational models of altruistic decision‐making and social motives: Advances, pitfalls, and future directions

“…Comprehensive theoretical and practical guidelines on preregistration: “Robust modeling in Cognitive Science”—Lee et al ( 2019 ) “Robust standards in Cognitive Science”—Crüwell et al ( 2019 ) “Preregistration in complex contexts: A preregistration template for the application of cognitive models”—Crüwell and Evans ( 2019 ) Diverse discussions on best practices and preregistration (e.g., points of agreement and disagreement, practical extensions, or theoretical considerations): Back‐to‐back special issues of Computational Brain and Behavior Blog post Popular Repositories commonly used: OSF —most popular across research disciplines AsPredicted —mainly used within psychology AEA —mainly used within economics …”

“…“Preregistration in complex contexts: A preregistration template for the application of cognitive models”—Crüwell and Evans ( 2019 )…”