Bayesian vs frequentist: comparing Bayesian model selection with a frequentist approach using the iterative smoothing method

Abstract

We have developed a frequentist approach for model selection which determines the consistency between any cosmological model and the data using the distribution of likelihoods from the iterative smoothing method. Using this approach, we have shown how confidently we can conclude whether the data support any given model without comparison to a different one. In this current work, we compare our approach with the conventional Bayesian approach based on the estimation of the Bayesian evidence using nested sampling. We use simulated future Roman (formerly WFIRST)-like type Ia supernovae data in our analysis. We discuss the limits of the Bayesian approach for model selection and show how our proposed frequentist approach can perform better in the falsification of individual models. Namely, if the true model is among the candidates being tested in the Bayesian approach, that approach can select the correct model. If all of the options are false, then the Bayesian approach will select merely the least incorrect one. Our approach is designed for such a case and we can conclude that all of the models are false.

Extreme data compression for Bayesian model comparison

“…Note that we remain of the Bayesian view that the Bayes Factor can indicate a preference for one model over another, but not falsify models in absolute terms, and we are not seeking frequentist methods to do this, which typically rely on tail probabilities of p(data|M ) as opposed to posterior model probabilities, p(M |data), cf. [38][39][40].…”

Gaussian discriminators between $$\varLambda $$CDM and wCDM cosmologies using expansion data

“…The formalism for a Gaussian or flat prior has been presented in [30]. Moreover, the Bayesian model selection has been utilized in [31,32] to understand reliability of the Bayesian evidence. In this work, we consider three important cosmological models which are linear in their parameters and apply the GLM method to understand how precision of the expansion rate data affects the significance of the model discrimination through Bayesian evidence.…”