Ten simple rules for designing and conducting undergraduate replication projects

Abstract

Conducting a replication study is a valuable way for undergraduate students to learn about the scientific process and gain research experience. By promoting the evaluation of existing studies to confirm their reliability, replications play a unique, though often underappreciated, role in the scientific enterprise. Involving students early in this process can help make replication mainstream among the new generation of scientists. Beyond their benefit to science, replications also provide an invaluable learning ground for students, from encouraging the development of critical thinking to emphasizing the importance of details and honing research skills. In this piece, we outline 10 simple rules for designing and conducting undergraduate replication projects, from conceptualization to implementation and dissemination. We hope that these guidelines can help educators provide students with a meaningful and constructive pedagogical experience, without compromising the scientific value of the replication project, therefore ensuring robust, valuable contributions to our understanding of the world.

Replication Research and Metascience

“…Although exact replication is the goal in a direct replication study, it is not possible to achieve, as time and circumstances will have changed and will impact the replication study in ways it did not impact the original study. 1,4,5,7 The first step in designing a replication study is to find the protocol used in the original study. This is usually published in the manuscript.…”

“…It is not expected that even with the exact same sample size and a similar population, the replication study will result in the exact same finding as the original study. 4,5,7 The researcher is looking to determine whether there is any significant effect in the same direction as the original study, and that the effect size is similar in both studies. 4…”

“…Analysis 1,4,5 Remains the same Improved or different analysis-related improvements in design, sample, or variable operationalization Dependent upon intent of researcher, may be an exact replication or original analysis, or an improvement in analysis…”

Teaching feminist psychology epistemology, ontology, and methodology through conceptual replication research proposals.

“…In this assignment, I decided to focus on conceptual replications of social psychology studies rather than direct replication to enhance students' learning. While conducting direct replications with students may have pedagogical benefits (see Moreau & Wiebels, 2023), conceptual replications were selected for three main reasons. First, the assignment was designed to encourage critical thinking.…”

“…That is, students were able to change multiple aspects of the original study and still make the case that it constituted a conceptual replication. This prompted interesting and nuanced conversations in class about the extent to which a study must be informed by the original in order to constitute a "replication" study (e.g., see Moreau & Wiebels, 2023). To participate in these discussions, students required a good grasp of the intricacies within current debates The group proposed to address this question of double disadvantage but switch the focus to a study that comes from a social constructivist epistemology that prioritizes lived experiences of being a gay Black man in the workplace using qualitative methods.…”