Objective Bayesian Comparison of Constrained Analysis of Variance Models

Abstract

In the social sciences we are often interested in comparing models specified by parametric equality or inequality constraints. For instance, when examining three group means {µ 1 , µ 2 , µ 3 } through an analysis of variance (ANOVA), a model may specify that µ 1 < µ 2 < µ 3 , while another one may state that {µ 1 = µ 3 } < µ 2 , and finally a third model may instead suggest that all means are unrestricted. This is a challenging problem, because it involves a combination of non-nested models, as well as nested models having the same dimension.We adopt an objective Bayesian approach, and derive the posterior probability of each model under consideration. Our method is based on the intrinsic prior methodology, with suitably modifications to accommodate equality and inequality constraints. Focussing on normal ANOVA models, a comparative assessment is carried out through simulation studies, showing that correct model identification is possible even in situations where frequentist power is low. We also present an application to real data collected in a psychological experiment.

BFpack: Flexible Bayes Factor Testing of Scientific Theories in R

“…Other Bayes factors and priors could also be considered of course. For testing parameters under a regression model, intrinsic priors (Casella and Moreno 2006;Consonni and Paroli 2017), (hyper) g priors (Bayarri and Garcia-Donato 2007;Liang, Paulo, Molina, Clyde, and Berger 2008;Mulder, Berger, Pena, and Bayarri 2020a), or non-local priors (Johnson and Rossell 2010) could be specified. All these Bayes factors, including the ones implemented in BFpack, all abide the notion of minimal prior information (via different routes), and they are all consistent for the proposed testing problems, implying that the evidence for the true hypothesis goes to infinity as the information in the data grows.…”

“…This implies that small deviations from the test value are more likely a priori than large deviations and that negative deviations are equally likely a priori as positive deviations from the test value, similar as in Jeffreys (1961) recommendations for prior specifications. Note that other commonly used priors are also centered at the test value, such as intrinsic priors (Casella and Moreno 2006;Consonni and Paroli 2017), (hyper) g priors (Bayarri and Garcia-Donato 2007;Liang et al 2008;Mulder et al 2020a), or non-local priors (Johnson and Rossell 2010). For technical details on the derivation this default Bayes factor we refer the interested reader to Mulder and Olsson-Collentine (2019) for the univariate regression model, and to for the general multivariate normal model with multiple groups.…”

Objective Bayesian comparison of order-constrained models in contingency tables

“…The above procedure, which we name intrinsic-encompassing, was first presented in Consonni & Paroli (2017) with regard to the comparison of constrained ANOVA models. There is however a significant difference.…”

“…When the default prior, as in our examples, is already proper, we can assess the robustness of our inference by letting the fraction of prior sample size to actual sample size vary: if the result is always above a threshold deemed to represent sufficient evidence, then we can safely conclude that the result is robust. Our method can also deal with starting default improper priors however, such as the Jeffreys prior Beta(1/2, 1/2): in this case however we would recommend using conditionally intrinsic priors, as opposed to fully intrinsic priors: for details see Consonni & Paroli (2017).…”

Default Bayesian Model Selection of Constrained Multivariate Normal Linear Models

“…For this reason there has been an extensive development of so-called default or automatic Bayes factors where a small subset of the data is used for prior specification (e.g. Spiegelhalter & Smith, 1982;O'Hagan, 1995;Berger & Pericchi, 1996;Berger & Mortera, 1999;Moreno et al, 1998;Berger & Pericchi, 2004;Klugkist et al, 2005;Mulder et al, 2009;Rouder et al, 2009;Klugkist et al, 2010;Hoijtink, 2011;Gu et al, 2014Gu et al, , 2017Consonni & Paroli, 2017;Böing-Messing et al, 2017;Mulder & Fox, 2018;Mulder & Olsson-Collentine, 2019, and the references therein).…”