Registered report: Survey on attitudes and experiences regarding preregistration in psychological research

Abstract

Background Preregistration, the open science practice of specifying and registering details of a planned study prior to knowing the data, increases the transparency and reproducibility of research. Large-scale replication attempts for psychological results yielded shockingly low success rates and contributed to an increasing demand for open science practices among psychologists. However, preregistering one’s studies is still not the norm in the field. Here, we conducted a study to explore possible reasons for this discrepancy. Methods In a mixed-methods approach, we conducted an online survey assessing attitudes, motivations, and perceived obstacles with respect to preregistration. Respondents (N = 289) were psychological researchers that were recruited through their publications on Web of Science, PubMed, PSYNDEX, and PsycInfo, and preregistrations on OSF Registries. Based on the theory of planned behavior, we predicted that positive attitudes (moderated by the perceived importance of preregistration) as well as a favorable subjective norm and higher perceived behavioral control positively influence researchers’ intention to preregister (directional hypothesis 1). Furthermore, we expected an influence of research experience on attitudes and perceived motivations and obstacles regarding preregistration (non-directional hypothesis 2). We analyzed these hypotheses with multiple regression models and included preregistration experience as a control variable. Results Researchers’ attitudes, subjective norms, perceived behavioral control, and the perceived importance of preregistration significantly predicted researchers’ intention to use preregistration in the future (see hypothesis 1). Research experience influenced both researchers’ attitudes and their perception of motivations to preregister, but not the perception of obstacles (see hypothesis 2). Descriptive reports on researchers’ attitudes, motivations and obstacles regarding preregistration are provided. Discussion Many researchers had already preregistered and had a rather positive attitude toward preregistration. Nevertheless, several obstacles were identified that may be addressed to improve and foster preregistration.

Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation

“…Students also reported a range of positive potential benefits of preregistration, including heightened transparency, improved clarity with the dissertation data-analysis process, and reduction of the lure to engage in QRPs (e.g., p-hack their results to obtain significant findings). However, before preregistration is integrated into dissertations as standard, some key barriers should be considered, such as time pressures, perceived rigidity of preregistration, and need for adequate training, as other researchers have recently noted (Spitzer & Mueller, 2023). We hope that this study will contribute to the ongoing reappraisal of Open Science to progress conversations about the robustness, replicability, and reliability of psychological science.…”

“…Perceptions of supervisory support. Finally, given the literature that suggests that perceived supervisor support affects students' experiences of their dissertation research (Roberts & Seaman, 2018) and that supervisor belief affects preregistration behavior (Spitzer & Mueller, 2023), to assess students' perceptions of their supervisory support at Time 2, we used a 14-item measure of perceptions of supervisor support. This scale includes items such as "I am satisfied with the support I have received from my supervisor" and "My supervisor was knowledgeable about research design/process as related to my project."…”

The Benefits of Preregistration and Registered Reports

“…Preregistration also allows peers to evaluate any deviations from the original plan such as changes to the analysis plan or preregistered hypothesis that otherwise go unnoticed (Toth et al, 2021;. This improved transparency is often regarded as one of the positive aspects of preregistration (Spitzer & Mueller, 2023;Toth et al, 2021). At the same time, preregistrations in practice too often include vague hypotheses, or completely omit important details of the methods, procedures, and analysis plan.…”

Registered Reports in Child Development: Introduction to the Special Section

“…As such, it was not immediately clear that they would be applicable to research using different methods. Over the past 10 years, however, researchers have demonstrated the versatility of RRs, applying the format to secondary data (Syed & Donnellan, 2020), longitudinal designs (MacSweeney et al, 2023), systematic reviews (Henderson et al, 2022), qualitative methods (Karhulahti et al, 2023), mixed methods (Spitzer & Mueller, 2023), and many others (see Henderson & Chambers, 2022). What was once viewed as a niche format for hypothesis‐driven experimental research has proven itself to be a broadly applicable publishing model that has the potential to greatly reshape our scientific knowledge.…”