The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum

Abstract

The popularity and use of Bayesian methods have increased across many research domains. The current article demonstrates how some less familiar Bayesian methods can be used. Specifically, we applied expert elicitation, testing for prior-data conflicts, the Bayesian Truth Serum, and testing for replication effects via Bayes Factors in a series of four studies investigating the use of questionable research practices (QRPs). Scientifically fraudulent or unethical research practices have caused quite a stir in academia and beyond. Improving science starts with educating Ph.D. candidates: the scholars of tomorrow. In four studies concerning 765 Ph.D. candidates, we investigate whether Ph.D. candidates can differentiate between ethical and unethical or even fraudulent research practices. We probed the Ph.D.s’ willingness to publish research from such practices and tested whether this is influenced by (un)ethical behavior pressure from supervisors or peers. Furthermore, 36 academic leaders (deans, vice-deans, and heads of research) were interviewed and asked to predict what Ph.D.s would answer for different vignettes. Our study shows, and replicates, that some Ph.D. candidates are willing to publish results deriving from even blatant fraudulent behavior–data fabrication. Additionally, some academic leaders underestimated this behavior, which is alarming. Academic leaders have to keep in mind that Ph.D. candidates can be under more pressure than they realize and might be susceptible to using QRPs. As an inspiring example and to encourage others to make their Bayesian work reproducible, we published data, annotated scripts, and detailed output on the Open Science Framework (OSF).

Leaving academia: PhD attrition and unhealthy research environments

“…Young, relatively inexperienced researchers might also have difficulty speaking up against, or even recognizing questionable research practices. Outcomes of recent studies indicate that an alarming percentage of PhD candidates report intentions to commit fraud driven by supervisors’ norms [ 61 ]. Research integrity courses for or together with supervisors could help to open up discussions about disciplinary norms and set a basic level of understanding about (in)appropriate practices.…”

Nudges can be both autonomy-preserving and effective: evidence from a survey and quasi-field experiment

“…After the conditions, respondents were asked to evaluate the subjective nudge effectiveness. We used self-admission rates for our main analysis and added an adapted version of the Bayesian truth serum to increase the credibility of the given answers (Prelec, 2004;John et al, 2012Weaver & Prelec, 2013Frank et al, 2017;Van de Schoot et al, 2021Schoenegger, 2023. Supplementary Appendix C elaborates on the serum.…”

One finding is no finding: Toward a replication culture in family business research

“…By integrating replication in the training of aspiring scholars, it would require only minimal effort from supervising experienced researchers, while simultaneously providing "object lessons about the scientific process, the importance of reporting standards, and the value of openness" (Frank & Saxe, 2012, p. 600) to the next generation of family business scholars during the most "imprintable" stage of their career (cf. Everett & Earp, 2015;Van de Schoot et al, 2021).…”