Detecting and diagnosing prior and likelihood sensitivity with power-scaling

Abstract

Determining the sensitivity of the posterior to perturbations of the prior and likelihood is an important part of the Bayesian workflow. We introduce a practical and computationally efficient sensitivity analysis approach that is applicable to a wide range of models, based on power-scaling perturbations. We suggest a diagnostic based on this that can indicate the presence of prior-data conflict or likelihood noninformativity. The approach can be easily included in Bayesian workflows with minimal work by the model builder. We present the implementation of the approach in our new R package priorsense and demonstrate the workflow on case studies of real data.

Quantification of empirical determinacy: the impact of likelihood weighting on posterior location and spread in Bayesian meta-analysis estimated with JAGS and INLA

“…This property proved useful for the definition of pEDL and pEDS. Because likelihood weighting affects both the location and the spread of the marginal posterior distributions, the proposed method is better suited for the quantification of empirical determinacy than other methods that are focused only on the total impact [Kallioinen et al, 2021, Roos et al, 2021.…”

Quantification of Empirical Determinacy: The Impact of Likelihood Weighting on Posterior Location and Spread in Bayesian Meta-Analysis Estimated with JAGS and INLA

“…Because the interpretation of model parameters remains unchanged, priors remain the same for varying values of w. The weighted likelihood terminology used in this paper is motivated by the role that the weight parameter, the nonnegative scalar w, plays in the exponential family (McCullagh and Nelder, 1989;Clayton, 1996). Alternatively, a likelihood raised to a power can be referred to as "power-scaling of the likelihood" (Kallioinen et al, 2021).…”

Prior knowledge elicitation: The past, present, and future

“…For example, for simple parametric models and informative data, the likelihood can dominate the prior and the gain from prior elicitation could be negligible. Thus, in many cases it may be sensible to start with some common default priors or priors weakly informed by some summary statistics of the data (e.g., by centering and normalizing the covariate and target values in regression), and then assess the need for more careful prior elicitation using prior diagnostic tools (Kallioinen et al, 2021).…”

“…Finally, a prior elicitation workflow should include one step to assess that the incorporated information is actually useful and an evaluation of the sensitivity of the results to the prior choice, including possible prior-data conflicts (Depaoli et al, 2020;Lopes and Tobias, 2011;Al-Labadi and Evans, 2017;Evans and Moshonov, 2006;Reimherr et al, 2021;Berger, 1990;Berger et al, 1994;Canavos, 1975;Hill and Spall, 1994;Skene et al, 1986;Jacobi et al, 2018;Roos et al, 2015;Pérez et al, 2006;Giordano et al, 2018;Bornn et al, 2010;Ho, 2020;Kallioinen et al, 2021).…”

“…For example, it is difficult for humans to estimate tiny probabilities, which is reflected in the difficulties of determining the tail shape of the elicited prior. A bias in the elicited prior and too thin tails can lead to strong prior sensitivity or prior-data conflict (Al-Labadi and Evans, 2017;Evans and Moshonov, 2006;Kallioinen et al, 2021;Bürkner, 2021). On the other hand, thick tailed priors may lead to ignoring the otherwise correctly elicited prior information.…”