Synthetic Likelihood in Misspecified Models: Consequences and Corrections

Abstract

We analyse the behaviour of the synthetic likelihood (SL) method when the model generating the simulated data differs from the actual data generating process. One of the most common methods to obtain SL-based inferences is via the Bayesian posterior distribution, with this method often referred to as Bayesian synthetic likelihood (BSL). We demonstrate that when the model is misspecified, the BSL posterior can be poorly behaved, placing significant posterior mass on values of the model parameters that do not represent the true features observed in the data. Theoretical results demonstrate that in misspecified models the BSL posterior can display a wide range of behaviours depending on the level of model misspecification, including being asymptotically non-Gaussian. Our results suggest that a recently proposed robust BSL approach can ameliorate this behavior and deliver reliable posterior inference under model misspecification. We document all theoretical results using a simple running example.

Combined statistical-mechanistic modeling links ion channel genes to physiology of cortical neuron types

“…The posterior distribution allows to quantify the uncertainty in our model parameter estimates, in contrast to previous work using genetic algorithms 27,28 . As has been observed in other contexts [29][30][31][32] , we found that SBI failed for our HH model and our data set due to a small but systematic mismatch between the data and the model, which could not be easily remedied by standard modifications to the HH model. We developed an algorithm that introduces noise to the summary statistics during training, which allowed it to perform reliable inference despite the model misspecification.…”

“…1b, c) -on average 18% of posterior-sampled parameter sets produced simulations with at least one undefined summary feature, such as undefined latency due to a lack of action potentials (Table S1, standard NPE). Therefore, we further investigated the reason for this failure and found that the poor performance of the SBI pipeline was due to a systematic mismatch between the electrophysiological recordings and simulations from the model, a phenomenon recently observed also in other settings [29][30][31][32] .…”

“…We achieved this by adding noise to the summary statistics derived from the simulations, effectively smoothing the feature space. In parallel work to this paper, this phenomenon has recently received more widespread attention [29][30][31][32] : for instance, robust neural posterior estimation 29 takes the opposite approach to our strategy and denoises the measured data towards the model using Monte-Carlo sampling. On the other hand, robust synthetic likelihood approaches 40 that estimate likelihoods rather than posteriors, work similarly to our approach.…”