Preregistration of secondary data analysis: A template and tutorial

Abstract

Preregistration has been lauded as one of the solutions to the so-called ‘crisis of confidence’ in the social sciences and has therefore gained popularity in recent years. However, the current guidelines for preregistration have been developed primarily for studies where new data will be collected. Yet, preregistering secondary data analyses---where new analyses are proposed for existing data---is just as important, given that researchers’ hypotheses and analyses may be biased by their prior knowledge of the data. The need for proper guidance in this area is especially desirable now that data is increasingly shared publicly. In this tutorial, we present a template specifically designed for the preregistration of secondary data analyses and provide comments and a worked example that may help with using the template effectively. Through this illustration, we show that completing such a template is feasible, helps limit researcher degrees of freedom, and may make researchers more deliberate in their data selection and analysis efforts.

Making the Black Box Transparent: A Template and Tutorial for Registration of Studies Using Experience-Sampling Methods

“…Fortunately, the idea of registration of studies using preexisting data is becoming more accepted. For example, van den Akker et al (2019) and Mertens and Krypotos (2019) have created registration templates for studies using preexisting data sets, and there are a growing number of resources and publications addressing this issue (e.g., Weston et al, 2019). Terminology is also being extended to account for this variation on the traditional preregistration format.…”

“…Time- and date-stamped access records will also facilitate the use of preexisting ESM data sets for Registered Reports. (For an extensive discussion of the issues around registration and transparency of studies using preexisting data sets, see van den Akker et al, 2019, and Weston et al, 2019).…”

“…In this article, we walk through our key additions and modifications to the Preregistration Challenge template (Mellor et al, 2019), upon which our template is based. Following reviewer comments, we have also incorporated elements from the preregistration template for preexisting data (van den Akker et al, 2019), to reflect the fact that many ESM studies yield large, rich data sets, which are also used by other researchers for primary analysis and, over time, for secondary analysis. The expanding conceptualization of the registration continuum to include postregistrations of studies using archival data (Benning et al, 2019) also brings new opportunities in this regard.…”

Making ERP Research More Transparent: Guidelines for Preregistration

“…Another possibility is to preregister analysis plans of data that have already been collected but not accessed, i.e. secondary data analysis (Mertens and Krypotos, 2019;Van den Akker et al, 2019) . 6 A clear separation between exploratory and confirmatory analyses has direct implications on the interpretability of the reported results.…”

Registered Reports With Developmental and Secondary Data: Some Brief Observations and Introduction to the Special Issue

“…The preceding list is by no means meant to be exhaustive but rather a list of notable issues that came up through the course of this special issue and our other work on registered reports and preregistration. We urge readers to consult some of the more extensive articles describing preregistration and registered reports with secondary data (van den Akker et al, 2019; Weston et al, 2019).…”