Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes

Abstract

In recent years, the scientific community has called for improvements in the credibility, robustness and reproducibility of research, characterized by increased interest and promotion of open and transparent research practices. While progress has been positive, there is a lack of consideration about how this approach can be embedded into undergraduate and postgraduate research training. Specifically, a critical overview of the literature which investigates how integrating open and reproducible science may influence student outcomes is needed. In this paper, we provide the first critical review of literature surrounding the integration of open and reproducible scholarship into teaching and learning and its associated outcomes in students. Our review highlighted how embedding open and reproducible scholarship appears to be associated with (i) students' scientific literacies (i.e. students’ understanding of open research, consumption of science and the development of transferable skills); (ii) student engagement (i.e. motivation and engagement with learning, collaboration and engagement in open research) and (iii) students' attitudes towards science (i.e. trust in science and confidence in research findings). However, our review also identified a need for more robust and rigorous methods within pedagogical research, including more interventional and experimental evaluations of teaching practice. We discuss implications for teaching and learning scholarship.

Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation

“…Our findings also contribute to the case that Open Science should be embedded into higher education for improved student scientific literacy and confidence (for a review, see Pownall et al, 2023). In response to the UK House of Commons Science and Technology Committee's call for evidence of the contributors to research integrity, FORRT argued the importance of the pedagogical consequences of how students are taught, mentored, and supervised (Azevedo et al, 2021).…”

“…To our knowledge, this study is the first to use quasiexperimental methods to begin to investigate this research question. This study thus responds directly to the calls of Pownall et al (2023) to adopt the principles of Open Science (e.g., robust methodologies, preregistration, open data sharing, collaborative science) to pedagogical research about the value of Open Science. As Pownall et al noted, to date, the majority of evidence available to educators and scholars who wish to make decisions about the incorporation of Open Science into their pedagogy typically relies on anecdotal and local-level evaluations of practice, which lack control groups and the ability to draw broader conclusions.…”

A Delphi Study to Strengthen Research-Methods Training in Undergraduate Psychology Programs

“…Updated accreditation standards could include specific items, such as learning about replication and sources of bias, to create a more structured approach for critically assessing the psychology literature and other forms of information. This topic is linked to integrating the principles of open science into undergraduate education, which others have encouraged (e.g., Pennington, 2023; Pownall et al, 2023).…”

Teaching feminist psychology epistemology, ontology, and methodology through conceptual replication research proposals.

“…Furthermore, running parallel to the literature that considers how to teach students about feminist approaches to psychology is a growing body of scholarship that explores how best to engage students with recent reappraisals of psychology’s replicability, robustness, and reproducibility (Pownall et al, 2023). Indeed, in recent years, psychologists have highlighted the problematic and questionable research practices that pervade the psychological literature (John et al, 2012) and have expressed concerns over the low rates of replicability and reproducibility of prominent psychology studies (Open Science Collaboration, 2015).…”

“…Educators have proposed many innovative methods to teach students about recent efforts to improve the robustness of psychological science. These have included, for example, in-class demonstrations, modules about research integrity, research experience internships, and discussions of metascience articles (see Pownall et al, 2023, for a review). Replication studies (i.e., the process of retesting a study’s research question with newly collected data) have been proposed as a particularly powerful way to help students to foster an appreciation of efforts to improve the psychology research (e.g., Jekel et al, 2020).…”