Preregistering Qualitative Research: A Delphi Study

Abstract

Preregistrations—records made a priori about study designs and analysis plans and placed in open repositories—are thought to strengthen the credibility and transparency of research. Different authors have put forth arguments in favor of introducing this practice in qualitative research and made suggestions for what to include in a qualitative preregistration form. The goal of this study was to gauge and understand what parts of preregistration templates qualitative researchers would find helpful and informative. We used an online Delphi study design consisting of two rounds with feedback reports in between. In total, 48 researchers participated (response rate: 16%). In round 1, panelists considered 14 proposed items relevant to include in the preregistration form, but two items had relevance scores just below our predefined criterion (68%) with mixed argument and were put forth again. We combined items where possible, leading to 11 revised items. In round 2, panelists agreed on including the two remaining items. Panelists also converged on suggested terminology and elaborations, except for two terms for which they provided clear arguments. The result is an agreement-based form for the preregistration of qualitative studies that consists of 13 items. The form will be made available as a registration option on Open Science Framework (osf.io). We believe it is important to assure that the strength of qualitative research, which is its flexibility to adapt, adjust and respond, is not lost in preregistration. The preregistration should provide a systematic starting point.

Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study

“…We sent invites to most participants directly, rather than informing them about the study beforehand, which likely explains the lower response rates. This explanation is supported by the fact that two recent Delphi studies using the same means of reporting their response rates had similar figures to ours (Haven et al, 2020;Mokkink et al, 2020). The possibility that the response rate introduces bias into the study is unlikely to be high, given the diversity in the study participants.…”

Improving the Credibility of Empirical Legal Research: Practical Suggestions for Researchers, Journals and Law Schools

“…One group recently developed a preregistration form through a consensus-based process (the same process used to create the checklist for behavioural studies above), participated by many researchers in the field. 136 Qualitative preregistrations may include details about the research team's background as it relates to the study as a form of reflexivity (e.g., attending to the experiences, positions and potential biases the researchers bring to bear on what they are studying). They may further include research questions and aims, planned procedures, sampling plans, data collection procedures, planned evidence criteria and triangulation, auditing and validation plans.…”

Open Science and Multicultural Research: Some Data, Considerations, and Recommendations

“…Important EM/D research outlets already recognize and incentivize certain OS practices (Lee, 2017). Specific practices (e.g., (pre)registration, material sharing, and archiving preprints) are applicable to investigators using experimental designs, original and archival observational research, qualitative inquiry and mixed methods, and CBPR (Haven et al, 2020;Lui et al, 2021a;Reischer & Cowan, 2020;van't Veer & Giner-Sorolla, 2016). Transparency in the knowledge-production process and equitable access to data/findings allow researchers to make contributions beyond the limits of their own resources.…”

“…In essence, student project proposals can take the form of Stage 1 Registered Reports and the completed manuscripts summarizing results and conclusions take the form of Stage 2 Registered Reports (Lui, 2020) (Benning et al, 2019;Tackett et al, 2019b). Although qualitative studies often do not test hypotheses by deductive, quantitative analyses, researchers still can prespecify the types of their research aim, anticipated duration of the planned investigation, details regarding study design, recruitment methods and justification for stopping criteria for data collection, process of data analysis and strategies for establishing credibility (Haven et al, 2020).…”

“…To reduce researcher onerousness, (pre)registration templates and recommendations are available-and rapidly growing-for diverse methods, including clinical trials (see ClinicalTrials.gov), social psychology experiments, qualitative and mixed methods research, secondary data analyses, experience sampling methods/ecological momentary assessment research, and systematic reviews (Haven et al, 2020;Kirtley et al, 2021;Krypotos et al, 2019;van't Veer & Giner-Sorolla, 2016;Weston et al, 2019). Many of these templates are stored in Open Science Framework or PROSPERO.…”