Advantages masquerading as “issues” in Bayesian hypothesis testing: A commentary on Tendeiro and Kiers (2019).

Abstract

Tendeiro and Kiers (2019) provide a detailed and scholarly critique of Null Hypothesis Bayesian Testing (NHBT) and its central component -the Bayes factor-that allows researchers to update knowledge and quantify statistical evidence. Tendeiro and Kiers conclude that NHBT constitutes an improvement over frequentist p-values, but primarily elaborate on a list of eleven 'issues' of NHBT. We believe that several issues identified by Tendeiro and Kiers are of central importance for elucidating the complementary roles of hypothesis testing versus parameter estimation and for appreciating the virtue of statistical thinking over conducting statistical rituals. But although we agree with many of their thoughtful recommendations, we believe that Tendeiro and Kiers are overly pessimistic, and that several of their 'issues' with NHBT may in fact be conceived as pronounced advantages. We illustrate our arguments with simple, concrete examples and end with a critical discussion of one of the recommendations by Tendeiro and Kiers, which is that "estimation of the full posterior distribution offers a more complete picture" than a Bayes factor hypothesis test.

Workflow techniques for the robust use of bayes factors.

“…In previous work, Tendeiro and Kiers (2019) have pointed out several issues with Bayes factors, and conclude that displaying posterior distributions may be the better way to go for Bayesian analyses. van Ravenzwaaij and Wagenmakers (2021) have responded to these issues and argue that “only Bayes factors can address the key question common to most empirical research in psychology: ‘to what extent do the data support the hypothesis that there is an effect’” (van Ravenzwaaij & Wagenmakers, 2021, p. 38). Tendeiro and Kiers (2021) have responded to van Ravenzwaaij and Wagenmakers (2021) and discuss their points critically with the goal to contribute “toward a better understanding among psychologists of null hypothesis Bayesian testing” (Tendeiro & Kiers, 2021, p. 2).…”

Efficient alternatives for Bayesian hypothesis tests in psychology.

“…The comments of van Ravenzwaaij and Wagenmakers (2019) illustrate this principle well. If only animals larger than cats are considered—so that the hypotheses are well separated—then one can test “no animal present” versus “animal present” very quickly.…”

“…With regard to the latter, they cite Johnson and Rossell (2010) and express concern that evidence is accumulated asymmetrically in favor of the alternative model. van Ravenzwaaij and Wagenmakers (2019) correctly point out that the claim that something is absent is more difficult to support than the claim that something is present, at least when one is uncertain about the size of the phenomenon that is present. They consider, for instance, the following null hypothesis.…”

Data aggregation can lead to biased inferences in Bayesian linear mixed models and Bayesian ANOVA: A simulation study

“…If item variance can in principle be present in a data set, it should be explicitly modeled using non-aggregated Bayesian hierarchical / linear-mixed effects models, software packages for which are easily accessible today (Bürkner, 2017;Carpenter et al, 2017). Bayesian modeling, and (null hypothesis) Bayes factor analyses have many advantages over frequentist p-values (Rouder et al, 2018;Tendeiro & Kiers, 2019;van Ravenzwaaij & Wagenmakers, 2021;Wagenmakers et al, 2010). However, some of the statistical caveats learned from frequentist tools need to be considered even when running Bayesian models.…”

“…Bayes factors can be used to test the null hypothesis that some effect is absent (i.e., that a parameter is zero) against an alternative hypothesis that the effect exists (i.e., that a parameter is different from zero), where Bayes factors perform the test under some prior assumed effect size. Bayes factor null hypothesis tests arguably provide a better alternative to frequentist p-values (Jeffreys, 1939;Kass & Raftery, 1995;Oberauer, 2022;Rouder, Haaf, & Vandekerckhove, 2018;Schad, Nicenboim, Bürkner, Betancourt, & Vasishth, 2022;Tendeiro & Kiers, 2019van Doorn, Aust, Haaf, Stefan, & Wagenmakers, 2021;van Ravenzwaaij & Wagenmakers, 2021;Wagenmakers, Lodewyckx, Kuriyal, & Grasman, 2010). In recent years, software has been developed that allows easy access to Bayesian hypothesis testing for lay users, such as the Bayesian analysis software WinBUGS (Lunn, Thomas, Best, & Spiegelhalter, 2000), JAGS (Plummer, 2003), PyMC3 (Salvatier, Wiecki, & Fonnesbeck, 2016), Stan (Carpenter et al, 2017), Turing (Ge, Xu, & Ghahramani, 2018), JASP (JASP Team, 2022), and others.…”