Questionable research practices following pre-registration

Abstract

The credibility of psychological findings can be undermined by a history of questionable research practices (QRPs) of researchers. One remedy for this problem is the use of the pre-registration of studies in which a research protocol is registered before the experiment starts. However, the current style of pre-registration can be negatively affected by other QRPs. The purpose of this paper was to demonstrate the ability to engage in QRPs even after a study has been pre-registered. In the demonstration study, we used a total of eight QRPs to obtain statistically meaningful results that supported an ad hoc hypothesis. Major system updates such as pre-registration, peer review, and evaluation are required to address these harmful practices. We hope that the present demonstration study gives momentum to further discussions on next-generation research practices.

Preregistration Is Neither Sufficient nor Necessary for Good Science

“…A major shortcoming of preregistration as a normative standard is that the increased transparency it provides may be more illusory than real. Under present systems of preregistration, there is still substantial room for selective reporting and researchers’ degrees of freedom (see Ikeda, Xu, Fuji, Zhu, & Yamada, 2019). For example, researchers may select which particular study or hypothesis to preregister; they may preregister a study only after running a long series of calibration tests that are not reported; they may preregister multiple studies or versions of the same study, then only report a selection; they may intentionally or unintentionally specify the hypotheses to be tested and analyses to be performed in loose terms, hence buying themselves flexibility in subsequent testing and analyses.…”

Ethical Consistency and Experience: An Attempt to Influence Researcher Attitudes Toward Questionable Research Practices Through Reading Prompts

“…But not all study designs permit blinding, and neither is preregistration is a cure-all. Apart from being vulnerable to being hacked by researchers with devious intent, preregistration does not preclude the unintended use of QRPs (Ikeda et al, 2020). Also, absent systemic pressures, researchers can be surprisingly recalcitrant about making the extra efforts required by preregistration and open access (Washburn et al, 2018).…”

Micropublishing During and After the COVID-19 Era

“…Moreover, the authors of each micro paper can be different. I have proposed a division of labor between the pre-registration and experimental groups in Registered Reports (Ikeda, Xu, Fuji, Zhu, & Yamada, 2019;Yamada, 2018). This format of micropublication is an easy way to implement this recommendation.…”