Detecting and diagnosing prior and likelihood sensitivity with power-scaling

Abstract

Determining the sensitivity of the posterior to perturbations of the prior and likelihood is an important part of the Bayesian workflow. We introduce a practical and computationally efficient sensitivity analysis approach using importance sampling to estimate properties of posteriors resulting from power-scaling the prior or likelihood. On this basis, we suggest a diagnostic that can indicate the presence of prior-data conflict or likelihood noninformativity and discuss limitations to this power-scaling approach. The approach can be easily included in Bayesian workflows with minimal effort by the model builder and we present an implementation in our new R package . We further demonstrate the workflow on case studies of real data using models varying in complexity from simple linear models to Gaussian process models.

Underestimation of Anaerobic Decomposition Rates in Sphagnum Litterbag Experiments by the Holocene Peatland Model Depends on Initial Leaching Losses

“…We used power-scaling of the prior and likelihood distributions as implemented in the priorsense package (0.0.0.9000) (Kallioinen et al, 2024) to analyze the relative sensitivity of the posterior distribution to small perturbations of the prior and likelihood in HPMe-LE-peat-l0 for HPM parameters and peat properties. This is a computationally nonexpensive way to check whether the data provide information about a parameter and where prior and data may provide conflicting information (Kallioinen et al, 2024). Results of this analysis and further information on the data analysis are shown in supporting information S3.…”

Supplementary material to "Underestimation of Anaerobic Decomposition Rates in Sphagnum Litterbag Experiments by the Holocene Peatland Model Depends on Initial Leaching Losses"

“…Power-scaling Power-scaling exponentiates prior (to analyze prior sensitivity) or likelihood (to analyze likelihood sensitivity) distributions by different constants 𝛼 > 0, where 𝛼 > 1 means that the scaled component gets more important relative to the other component, and 𝛼 < 1 means it gets less important (Kallioinen et al, 2024). We varied 𝛼 from 0.99 to 1.01 (default option) and identified sensitivity with the cumulative Jensen-Shannon distance and a threshold of 0.05, as suggested in Kallioinen et al (2024).…”

priorsense: Prior Diagnostics and Sensitivity Analysis

“…The main diagnostic function provided by priorsense is powerscale_sensitivity. Given a fitted model or draws object, it computes the powerscaling sensitivity diagnostic described in Kallioinen et al (2023). It does so by perturbing the prior and likelihood and computing the effect on the posterior, without needing to refit the model (using Pareto smoothed importance sampling and importance weighted moment matching; Vehtari et al 2022).…”