Researcher Degrees of Freedom in the Psychology of Religion

Abstract

There is a push in psychology toward more transparent practices, stemming partially as a response to the replication crisis. We argue that the psychology of religion should help lead the way toward these new, more transparent practices to ensure a robust and dynamic subfield. One of the major issues that proponents of Open Science practices hope to address is researcher degrees of freedom (RDF). We pre-registered and conducted a systematic review of the 2017 issues from three psychology of religion journals. We aimed to identify the extent to which the psychology of religion has embraced Open Science practices and the role of RDF within the subfield. We found that many of the methodologies that help to increase transparency, such as pre-registration, have yet to be adopted by those in the subfield. In light of these findings, we present recommendations for addressing the issue of transparency in the psychology of religion and outline how to move toward these new Open Science practices.

United on Sunday: The effects of secular rituals on social bonding and affect

“…Firstly, this study was not pre-registered. Given the changes suggested by those promoting Open Science methodologies since the advent of the replication crisis [ 80 – 82 ], the methods and analysis plans could have been registered in advance of conducting the study. Though pre-registration was not done in this case, the full anonymised dataset and the research materials are provided in supplementary materials in accordance with other Open Science practices, and a power analysis was provided to support the sample size used in this study.…”

Harmony Between Humanity and Nature: Natural Vs. Synthetic Drug Preference in Chinese Atheists and Taoists

“…Sample sizes varied according to the availability of participants. To control for the opportunistic use of Research Degrees of Freedom which may tweak the data and increase the false positive rate in experiments (Charles et al, 2019 ), we did not perform any statistical analysis of the results until all data collection was completed, and no additional data were added after the initial analyses.…”

Absorption, Mentalizing, and Mysticism

“…The present studies were designed as 'exploratory research.' In contrast to confirmatory research, which aims to rigorously test specific hypotheses from an established method or theory, the aim of exploratory research is broader and used to generate new directions and hypotheses for further investigation (for a full discussion see, Charles, Bartlett, Messick, Coleman and Uzdavines 2018;de Groot 1954de Groot /2014. This is an important distinction to declare because it affects how our research was designed (for example, favoring a broad research question over very specific hypotheses) and thus how it should be interpreted.…”

“…This is an important distinction to declare because it affects how our research was designed (for example, favoring a broad research question over very specific hypotheses) and thus how it should be interpreted. Moreover, although this distinction is always implicit in study design, that researchers fail to make this explicit is a major methodological limitation of research in general, and within the psychology of religion in specific (Charles et al 2018).…”