Comparison of statistical sampling methods with ScannerBit, the GAMBIT scanning module

Abstract

We introduce ScannerBit, the statistics and sampling module of the public, open-source global fitting framework GAMBIT. ScannerBit provides a standardised interface to different sampling algorithms, enabling the use and comparison of multiple computational methods for inferring profile likelihoods, Bayesian posteriors, and other statistical quantities. The current version offers random, grid, raster, nested sampling, differential evolution, Markov Chain Monte Carlo (MCMC) and ensemble Monte Carlo samplers. We also announce the release of a new standalone differential evolution sampler, Diver, and describe its design, usage and interface to ScannerBit. We subject Diver and three other samplers (the nested sampler MultiNest, the MCMC GreAT, and the native ScannerBit implementation of the ensemble Monte Carlo algorithm T-Walk) to a battery of statistical tests. For this we use a realistic physical likelihood function, based on the scalar singlet model of dark matter. We examine the performance of each sampler as a function of its adjustable settings, and the dimensionality of the sampling problem. We evaluate performance on four metrics: optimality of the best fit found, completeness in exploring the best-fit region, number of likelihood evaluations, and total runtime. For Bayesian posterior estimation at high resolution, T-Walk provides the most accurate and timely mapping of the full parameter space. For profile likelihood analysis in less than about ten dimensions, we find that Diver and MultiNest score similarly in terms of best fit and speed, outperforming GreAT a

FlavBit: a GAMBIT module for computing flavour observables and likelihoods

“…It determines the off-diagonal terms on a case-bycase basis in each likelihood function, in order to make it possible for different likelihood functions to adjust the correlations according to whether different nuisance parameters are scanned over directly, or should be included via the correlation matrix. 8 FlavBit builds the full covariance matrix by summing the experimental and theoretical covariance matrices. If an observable and its measurements are uncorrelated with other observables, the resulting uncertainty then becomes simply the sum in quadrature of the theoretical and experimental errors.…”

“…8 Users of FlavBit should be aware of a potential pitfall arising from this arrangement. The theory uncertainties and correlations that we include in the current release and describe in this paper already incorporate uncertainties on input parameters such as form factors, decay constants, SM masses and couplings, and in particular, CKM matrix entries.…”

“…The electroweak penguin likelihood in FlavBit is calculated using the angular observables of the B 0 → K * 0 μ + μ − decay, as measured by LHCb [47] in dimuon invariant mass squared bins of (1.1, 2.5), (2.5, 4), (4,6), (6,8), (15,17) and (17,19) GeV 2 . The bin (11, 12.5) GeV 2 cannot be used in the likelihood, as the relative phase between the charmonium resonances in this bin and the non-resonant decay is not currently known.…”

“…FlavBit takes into account all theoretical and experimental correlations between the different observables. The resulting likelihoods can be incorporated into the GAMBIT global likelihood to scan the parameter spaces of various models for new physics [4,[8][9][10][11], taking into account complementary constraints from direct production [12], dark matter searches [13], and SM and related precision measurements [14].…”

Robustness of dark matter constraints and interplay with collider searches for New Physics

“…We then apply different constraints from the dark matter and collider experiments, which are described below. We do not intend to perform a robust statistical analysis as performed for example by the GAMBIT Collaboration [13,118,119].…”