Systemic problems require systemic solutions: the need for coordination and cooperation to improve research quality

Abstract

Various factors contribute to low reproducibility and replicability of scientific findings. Whilst not all of these are necessarily problematic, there is growing acceptance that there is room for improvement. Many sectoral organisations have a role to play in this, by refining incentives and rewards, promoting specific behaviours such as open research practices, and exploring innovations in grant funding and scientific publishing. However, given the systems nature of the challenge, real change will require the coordination of these efforts, and partnerships that ensure alignment of activity and interoperability of training. Efforts to improve research quality will require investment, in infrastructure, training, and research on research to ensure that innovative solutions are evidence-based, and potential unintended consequences are explored (and avoided). National structures (e.g., the planned UK Committee on Research Integrity) should focus on understanding the research system, identifying areas for improvement, and promoting research to understand the impact of novel approaches and innovations, in order to advise on how to maximise benefit and avoid harm.

Epistemic Metadata for Computational Engineering Information Systems

“…The first three explanations do not provide a good excuse, they are rather indicative of careless or egoistic behaviour -scientific communities can justifiably take action against them: No. 1 does not work, since there is so little agreement on the ordinary level of reproducibility that we cannot just take its understanding for granted; researchers may be unaware of this, but institutions, organizations, projects, and bottom-up initiatives are already raising that awareness [18] and will need to continue to do so. No.…”

“…Reproducibility is known to be one of the major challenges to open science and the scientific process at large [16]; e.g., examining applied mathematics papers, Riedel et al [17] find the reported results to be completely reproducible in only four out of 108 cases, and partially reproducible in only one other case. To advance on this challenge, reproducibility networks are being formed, for which the RIOT principles (reproducibility, interpretability, openness, transparency [18]) were formulated as guidelines, complementing the FAIR principles. Unfortunately, when working toward an improved reproducibility and compliance with the FAIR principles, communities risk overshooting by formulating excessive demands on the data provenance (i.e., origin and genesis) documentation.…”

Music and neuroscience research for mental health, cognition, and development: Ways forward

“…Whilst not all of these factors are necessarily problematic, we believe there is “room” for improvement and development in this research field. For this reason, we propose some state-of-the-art approaches to minimizing the frequency of commonly observed limitations, based on potential solutions that are observed to be universal in empirical research (Lin et al, 2011 ; Boutron and Ravaud, 2018 ; Brown et al, 2018 ; Jaschke et al, 2018 ; Sala and Gobet, 2020 ; Gustavson et al, 2021 ; Ganley et al, 2022 ), and likewise can be applied to this specific domain, too.…”

“…In another way, proper funding and personnel for more pilot and feasibility studies should be important, minimizing many small, non-randomized studies with cross-sectional survey data, and a variety of non-validated questionnaires (Brown et al, 2018 ). Also, partnerships across researchers in academia and research organizations outside of academia (e.g., music industries) or private companies should result in the reassurance of data integrity and results in quality generated in this domain (Ganley et al, 2022 ).…”

Seminatural environments for rodent behavioral testing: a representative design improving animal welfare and enhancing replicability

“…The remedies for low reproducibility is thought to be enhanced scientific rigor, meaning that, for example, statistical methods should be strengthened, the analysis plan should be prepublished, collaboration across labs should be stimulated, data should be made openly available, and detailed experimental procedures should be reported ( Munafo et al, 2017 ; Stevens, 2017 ; Ganley et al, 2022 ; Lu and Daugherty, 2022 ).…”