The replication crisis has led to positive structural, procedural, and community changes

Abstract

The emergence of large-scale replication projects yielding successful rates substantially lower than expected caused the behavioural, cognitive, and social sciences to experience a so-called ‘replication crisis’. In this Perspective, we reframe this ‘crisis’ through the lens of a credibility revolution, focusing on positive structural, procedural and community-driven changes. Second, we outline a path to expand ongoing advances and improvements. The credibility revolution has been an impetus to several substantive changes which will have a positive, long-term impact on our research environment.

Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation

“…These efforts aim to strengthen student knowledge and engagement in research to become more savvy consumers of science (Korbmacher et al, 2023). There is now a need for researchers to continue this line of work, critically and empirically investigating how barriers to Open Science can be negated with students (and, indeed, more broadly) to continue embedding high-quality, rigorous, thoughtful research practices into the undergraduate dissertation and beyond.…”

Reducing intervention- and research-induced inequalities to tackle the digital divide in health promotion

“…Similarly, to aid transparency around addressing inequalities, existing reporting guidelines such as CONSORT or STROBE should be extended to include social inequality indicators in baseline assessments and for potential differences to be explicitly checked when analyzing the data, e.g., through sensitivity analyses [ 8 ]. Since many journals require reporting checklists to be included upon submission, and preregistration of studies is becoming more common (even for study designs other than clinical trials), it is likely that these structural changes will lead to a relatively quick change in research practices [ 14 ]. Finally, researchers should involve key stakeholders, including patients, clinicians, community members, and potential end-users of digital health interventions, in the study planning process as early as possible (e.g., through community-based participatory research or patient and public involvement [ 4 ], to address research questions that are relevant and important to them and to design study materials in accordance with their needs and expectations.…”

“…Yet, these projects are rare and typically require funding by multinational organizations (e.g., the European Union, which is still largely limited to EU countries). This issue may be overcome through big team science such as Many Labs projects (see [ 14 ], for a discussion), which typically involve hundreds of researchers from various countries who pool their resources (personnel and study funding) to collect data using the same (translated) measures, and potentially also conduct experiments or interventions using the same (translated) materials. In this vein, effects may be compared between countries and cultures to see whether effects are indeed generalizable.…”

The Replication Database: Documenting the Replicability of Psychological Science

“…Such attempts have identified relatively low replication rates (<60%; Camerer et al, 2016;Klein et al, 2014;Klein et al, 2018;Open Science Collaboration, 2015) with few exceptions (Protzko et al, 2024;Soto, 2019). These findings have motivated claims that the psychological sciences are suffering from a 'replication crisis' (Maxwell et al, 2015;Nelson et al, 2018;Schooler, 2014) and are now undergoing a 'credibility revolution' (Korbmacher et al, 2023;Vazire, 2018). Concerns about replicability have therefore grown over the last decade, and have also been echoed in other sciences (e.g., Errington et al, 2021;Nosek & Errington, 2017).…”