Simple Bayesian testing of scientific expectations in linear regression models

The American Statistician

Wagenmakers

Marsman

2020

The Savage-Dickey density ratio is a specific expression of the Bayes factor when testing a precise (equality constrained) hypothesis against an unrestricted alternative. The expression greatly simplifies the computation of the Bayes factor at the cost of assuming a specific form of the prior under the precise hypothesis as a function of the unrestricted prior. A generalization was proposed by Verdinelli and Wasserman such that the priors can be freely specified under both hypotheses while keeping the computational advantage. This article presents an extension of this generalization when the hypothesis has equality as well as order constraints on the parameters of interest. The methodology is used for a constrained multivariate t-test using the JZS Bayes factor and a constrained hypothesis test under the multinomial model.

Section: Introductionmentioning

confidence: 99%

A Generalization of the Savage–Dickey Density Ratio for Testing Equality and Order Constrained Hypotheses

The American Statistician

Wagenmakers

Marsman

2020

“…In the case of testing location parameters and group variances (Böing-Messing et al 2017a), generalized adjusted fractional Bayes factors are used based on minimal fractions under all groups where the implicit fractional prior is adjusted to the boundary of the constrained space (e.g., to the test value for a Bayesian t test). The fractional prior is located to the boundary of the constrained space to abide the rational that small effects are more plausible a priori than large effects (typical in applied research) and that negative effects are equally plausible as positive effects (Mulder 2014;Mulder and Olsson-Collentine 2019). The default Bayes factor is fully automatic for a given set of constrained hypotheses, and thus a prior scale of the effects does not need to be specified based on prior expectations about the anticipated effects.…”

Section: Theoretical Background Of Bayesian Statistical Inferencementioning

confidence: 99%

“…Note that other commonly used priors are also centered at the test value, such as intrinsic priors (Casella and Moreno 2006;Consonni and Paroli 2017), (hyper) g priors (Bayarri and Garcia-Donato 2007;Liang et al 2008;Mulder et al 2020a), or non-local priors (Johnson and Rossell 2010). For technical details on the derivation this default Bayes factor we refer the interested reader to Mulder and Olsson-Collentine (2019) for the univariate regression model, and to for the general multivariate normal model with multiple groups. Furthermore, for univariate models the probability densities in the first factor in Equation 7 can be computed using the dmvt() function in the mvtnorm package (Genz et al 2021), and the probabilities in the second factor can be computed using the pmvt() function in the mvtnorm package.…”

Section: A Technical and Computational Detailsmentioning

confidence: 99%

BFpack: Flexible Bayes Factor Testing of Scientific Theories in R

Williams

et al. 2021

J. Stat. Soft.

There have been considerable methodological developments of Bayes factors for hypothesis testing in the social and behavioral sciences, and related fields. This development is due to the flexibility of the Bayes factor for testing multiple hypotheses simultaneously, the ability to test complex hypotheses involving equality as well as order constraints on the parameters of interest, and the interpretability of the outcome as the weight of evidence provided by the data in support of competing scientific theories. The available software tools for Bayesian hypothesis testing are still limited however. In this paper we present a new R package called BFpack that contains functions for Bayes factor hypothesis testing for the many common testing problems. The software includes novel tools for (i) Bayesian exploratory testing (e.g., zero vs positive vs negative effects), (ii) Bayesian confirmatory testing (competing hypotheses with equality and/or order constraints), (iii) common statistical analyses, such as linear regression, generalized linear models, (multivariate) analysis of (co)variance, correlation analysis, and random intercept models, (iv) using default priors, and (v) while allowing data to contain missing observations that are missing at random.

“…As is well-known, the Bayes factor can be sensitive to the prior for testing equality constrained hypotheses (Jeffreys, 1961). To avoid ad hoc or arbitrary prior specification in the case of little prior information, we extend the adjusted fractional Bayes factor (Mulder, 2014b;Mulder & Olsson-Collentine, 2019) to the multivariate normal linear model. We start by using the (flat) noninformative improper Jeffrey prior, p N ðH, RÞ / jRj À Pþ1 2 : As this prior is improper, we follow O' Hagan (1995) and divide the marginal likelihood by a marginal likelihood where the likelihood is raised to a fraction 'b', so that the unknown normalizing constant of the prior cancels out, p t ðY, bÞ…”

Section: A Default Bayes Factor For Testing Constrained Hypothesesmentioning

confidence: 99%

Bayesian Testing of Scientific Expectations under Multivariate Normal Linear Models

Multivariate Behavioral Research

2021

The multivariate normal linear model is one of the most widely employed models for statistical inference in applied research. Special cases include (multivariate) t testing, (M)AN(C)OVA, (multivariate) multiple regression, and repeated measures analysis. Statistical criteria for a model selection problem where models may have equality as well as order constraints on the model parameters based on scientific expectations are limited however. This paper presents a default Bayes factor for this inference problem using fractional Bayes methodology. Group specific fractions are used to properly control prior information. Furthermore the fractional prior is centered on the boundary of the constrained space to properly evaluate order-constrained models. The criterion enjoys various important properties under a broad set of testing problems. The methodology is readily usable via the R package 'BFpack'. Applications from the social and medical sciences are provided to illustrate the methodology.