An integrative framework for planning and conducting Non-Intervention, Reproducible, and Open Systematic Reviews (NIRO-SR)

Abstract

Mounting evidence indicates issues with low adherence to existing consensus-based guidelines for conducting systematic reviews (SRs), meaning that SRs can be subject to selective or misreporting practices. This problem arises in part from scarce guidance for reproducible reporting practices. This is compounded by the fact that existing guidelines are mainly applicable to interventional research designs, with systematic reviewers of non-interventional studies resorting to customised tools that deviate from best practice. Here, we present the development of the first comprehensive tool for conducting and reporting Non-Interventional, Reproducible, and Open Systematic Reviews (NIRO-SR). NIRO-SR is a 68-item checklist composed of two parts that provide itemised guidance on the preparation of a protocol for pre-registration (Part A) and reporting the review (Part B) in a reproducible and transparent manner. This paper, the tool, and an open repository (https://osf.io/f3brw/) provide a comprehensive resource for anyone who aims to conduct a high quality SR of non-interventional studies.

A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000–2020)

“…Use well-established, up-to-date reporting guidelines intended for meta-analyses such as: the recently updated PRISMA 2020 (Page et al, 2021); the focused-on reliability generalization meta-analyses REGEMA (Sánchez-Meca et al, 2021); the focused-on non-intervention studies NIRO-SR (Topor et al, 2020), for example.…”

Introduction to PRISMA 2020 and implications for research synthesis methodologists

“…We are aware of 17 reporting guidelines for research synthesis developed outside of the PRISMA banner (Table 1), sourced from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network library and literature searches conducted to inform the update to PRISMA 2009 39–55 . These 17 guidelines span the disciplines of health, 39–42,44,49–52 social sciences, 41,42,44,52 psychology, 45,46,54,55 neuroscience, 47 economics 53 and conservation and environmental management 48 .…”

“…We are aware of 17 reporting guidelines for research synthesis developed outside of the PRISMA banner (Table 1), sourced from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network library and literature searches conducted to inform the update to PRISMA 2009 39–55 . These 17 guidelines span the disciplines of health, 39–42,44,49–52 social sciences, 41,42,44,52 psychology, 45,46,54,55 neuroscience, 47 economics 53 and conservation and environmental management 48 . The content of some of these guidelines overlaps considerably with PRISMA 2009, while several provide guidance on topics not addressed in PRISMA 2009 or its extensions, including synthesis of qualitative data, 40,46 realist synthesis, 41 and synthesis of the quantitative effects of interventions using alternative methods to meta‐analysis 52 .…”

Best Practices for Addressing Missing Data through Multiple Imputation

“…Typically, preregistration includes specifying research questions/hypotheses, the research design, and data analysis plan before conducting analyses (e.g. Mertens & Krypotos, 2019;Nosek et al, 2018;Pownall et al, 2021;Tierney et al, 2020Tierney et al, , 2021Topor et al, 2020). This process helps to avoid too many 'researcher degrees of freedom' leading to potentially spurious findings (Wicherts et al, 2016).…”