Mounting evidence indicates issues with low adherence to existing consensus-based guidelines for conducting systematic reviews (SRs), meaning that SRs can be subject to selective or misreporting practices. This problem arises in part from scarce guidance for reproducible reporting practices. This is compounded by the fact that existing guidelines are mainly applicable to interventional research designs, with systematic reviewers of non-interventional studies resorting to customised tools that deviate from best practice. Here, we present the development of the first comprehensive tool for conducting and reporting Non-Interventional, Reproducible, and Open Systematic Reviews (NIRO-SR). NIRO-SR is a 68-item checklist composed of two parts that provide itemised guidance on the preparation of a protocol for pre-registration (Part A) and reporting the review (Part B) in a reproducible and transparent manner. This paper, the tool, and an open repository (https://osf.io/f3brw/) provide a comprehensive resource for anyone who aims to conduct a high quality SR of non-interventional studies.
In recent years, the scientific community has called for improvements in the credibility, robustness, and reproducibility of research, characterized by higher standards of scientific evidence, increased interest in open practices, and promotion of transparency. While progress has been positive, there is a lack of consideration about how this approach can be embedded into undergraduate and postgraduate research training. Currently, the impact of integrating an open and reproducible approach into the curriculum on student outcomes is not well articulated in the literature. Therefore, in this paper, we provide the first comprehensivereview of how integrating open and reproducible scholarship into teaching and learning may impact students, using a large-scale, collaborative, team-science approach. Our review highlighted how embedding open and reproducible scholarship may impact: (1) students’ scientific literacies (i.e., students’ understanding of open research, consumption of science, and the development of transferable skills); (2) student engagement (i.e., motivation and engagement with learning, collaboration, and engagement in open research), and (3) students’attitudes towards science (i.e., trust in science and confidence in research findings). Our review also identified a need for more robust and rigorous methods within evaluations of teaching practice. We discuss implications for teaching and learning scholarship in this area.
Götz et al. (2021) argue that small effects are the indispensable foundation for a cumulative psychological science. Whilst we applaud their efforts to bring this important discussion to the forefront, we argue that their core arguments do not hold up under scrutiny, and if left uncorrected have the potential to undermine best practices in reporting and interpreting effect size estimates. Their article can be used as a convenient blanket defense to justify ‘small’ effects as meaningful. In our reply, we first argue that comparisons between psychological science and genetics are fundamentally flawed because these disciplines have vastly different goals and methodology. Second, we argue that p-values, not effect sizes, are the main currency for publication in psychology, meaning that any biases in the literature are caused by this pressure to publish statistically significant results, not a pressure to publish large effects. Third, we contend that claims regarding small effects as important and consequential must be supported by empirical evidence, or at least require a falsifiable line of reasoning. Finally, we propose that researchers should evaluate effect sizes in relative, not absolute terms, and provide several approaches of how this can be achieved.
This systematic review and meta-analysis investigated clinical and cognitive outcomes of external trigeminal nerve stimulation (eTNS) in neurological and mental health disorders (CRD42022322038). PubMed, OVID, Web of Science, Chinese National Knowledge Infrastructure, Wanfang, and VIP database for Chinese technical periodicals were searched (until 16/03/2022) to identify empirical trials investigating cognitive and clinical outcomes of eTNS in neurological or mental health disorders. The Cochrane Risk of Bias 2.0 tool assessed Randomized controlled trials (RCTs), while the Risk of Bias of Non-Randomised Studies (ROBINS-I) assessed single-arm trials. Fifty-three (26 RCTs; 27 single-arm) trials were included, of which 12 were meta-analyzed (N=1,048). The meta-analyses showed that migraine pain intensity (N=4, SMD=1.03, 95%CI[0.84-1.23]) and quality of life (N=2, SMD=1.88, 95%CI[1.22.5-2.53]) significantly improved with eTNS combined with anti-migraine medication. Dimensional measures of depression improved with eTNS across 3 different disorders (N=3, SMD=0.45, 95%CI[0.01-0.88]). eTNS was well-tolerated with a good adverse event profile across disorders. eTNS is potentially clinically relevant in other disorders, but well-blinded, adequately powered RCTs must replicate findings and establish optimal dosage guidance.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.