Many Labs projects have become the gold standard for assessing the replicability of key findings in psychological science. The Many Labs 4 project recently failed to replicate the mortality salience effect where being reminded of one’s own death strengthens the own cultural identity. Here, we provide a Bayesian reanalysis of Many Labs 4 using meta-analytic and hierarchical modeling approaches and model comparison with Bayes factors. In a multiverse analysis we assess the robustness of the results with varying data inclusion criteria and prior settings. Bayesian model comparison results largely converge to a common conclusion: We find evidence against a mortality salience effect across the majority of our analyses. Even when ignoring the Bayesian model comparison results we estimate overall effect sizes so small (between d = 0.03 and d = 0.18) that it renders the entire field of mortality salience studies as uninformative.

Statistical modeling is generally meant to describe patterns in data in service of the broader scientific goal of developing theories to explain those patterns. Statistical models support meaningful inferences when models are built so as to align parameters of the model with potential causal mechanisms and how they manifest in data. When statistical models are instead based on assumptions chosen by default, Attempts to draw inferences can be uninformative or even paradoxical—in essence, the tail is trying to wag the dog.These issues are illustrated by van Doorn et al. (in press) in the context of using BayesFactors to identify effects and interactions in linear mixed models. We show that the problems identified in their applications can be circumvented by using priors over inherently meaningful units instead of default priors on standardized scales. This case study illustrates how researchers must directly engage with a number of substantive issues in order to support meaningful inferences, of which we highlight two: The first is the problem of coordination, which requires a researcher to specify how the theoretical constructs postulated by a model are functionally related to observable variables. The second is the problem of generalization, which requires a researcher to consider how a model may represent theoretical constructs shared across similar but non-identical situations, along with the fact that model comparison metrics like Bayes Factors do not directly address this form of generalization. For statistical modeling to serve the goals of science, models cannot be based on default assumptions, but should instead be based on an understanding of their coordination function and on how they represent causal mechanisms that may be expected to generalize to other related scenarios.

Cognitive models provide a substantively meaningful quantitative description of latent cognitive processes. The quantitative formulation of these models supports cumulative theory building and enables strong empirical tests. However, the non-linearity of these models and pervasive correlations among model parameters pose special challenges when applying cognitive models to data. Firstly, estimating cognitive models typically requires large hierarchical data sets that need to be accommodated by an appropriate statistical structure within the model. Secondly, statistical inference needs to appropriately account for model uncertainty to avoid overconfidence and biased parameter estimates. In the present work we show how these challenges can be addressed through a combination of Bayesian hierarchical modelling and Bayesian model averaging. To illustrate these techniques, we apply the popular diffusion decision model to data from a collaborative selective influence study.

On Monday November 9th 2020, Pfizer and BioNTech issued a press release in which Pfizer CEO Dr. Albert Bourla stated: “Today is a great day for science and humanity. The first set of results from our Phase 3 COVID-19 vaccine trial provides the initial evidence of our vaccine’s ability to prevent COVID-19”. But what exactly are the initial data, and how strong is the initial evidence? Based on the available data, we conduct a Bayesian logistic regression with group membership (placebo vs. vaccine) as a predictor. Our results confirm that the Pfizer interim Phase 3 data are indeed highly promising. Even though the case numbers are relatively small, their distribution across the placebo and vaccinated group is so lopsided that (1) the evidence in favor of effectiveness is overwhelming; (2) the effect is almost certainly very large, although how large exactly is difficult to tell.

A recent trial assessed the efficacy of hydroxychloroquine (HCQ) in the treatment of patients with common coronavirus disease-19 (COVID-19). 1 The results showed that “On day 7, COVID-19 nucleic acid of throat swabs was negative in 13 (86.7%) cases [of out 15] in the HCQ group and 14 (93.3%) cases [out of 15] in the control group (P>0.05). Here we present a Bayesian reanalysis of the data, showing that the data under consideration neither undercut nor support the HCQ hypothesis in a compelling fashion. One of the reasons for this lack of diagnosticity is that both the HCQ group and the control group showed high recovery rates. In line with the authors’ conclusion, more data are needed to draw definite conclusions.