Bayesian model selection for group studies

Abstract

Bayesian model selection (BMS) is a powerful method for determining the most likely among a set of competing hypotheses about the mechanisms that generated observed data. BMS has recently found widespread application in neuroimaging, particularly in the context of dynamic causal modelling (DCM). However, so far, combining BMS results from several subjects has relied on simple (fixed effects) metrics, e.g. the group Bayes factor (GBF), that do not account for group heterogeneity or outliers. In this paper, we compare the GBF with two random effects methods for BMS at the between-subject or group level. These methods provide inference on model-space using a classical and Bayesian perspective respectively. First, a classical (frequentist) approach uses the log model evidence as a subject-specific summary statistic. This enables one to use analysis of variance to test for differences in log-evidences over models, relative to intersubject differences. We then consider the same problem in Bayesian terms and describe a novel hierarchical model, which is optimised to furnish a probability density on the models themselves. This new variational Bayes method rests on treating the model as a random variable and estimating the parameters of a Dirichlet distribution which describes the probabilities for all models considered. These probabilities then define a multinomial distribution over model space, allowing one to compute how likely it is that a specific model generated the data of a randomly chosen subject as well as the exceedance probability of one model being more likely than any other model. Using empirical and synthetic data, we show that optimising a conditional density of the model probabilities, given the log-evidences for each model over subjects, is more informative and appropriate than both the GBF and frequentist tests of the log-evidences. In particular, we found that the hierarchical Bayesian approach is considerably more robust than either of the other approaches in the presence of outliers. We expect that this new random effects method will prove useful for a wide range of group studies, not only in the context of DCM, but also for other modelling endeavours, e.g. comparing different source reconstruction methods for EEG/MEG or selecting among competing computational models of learning and decision-making.

Autonomic and brain responses associated with empathy deficits in autism spectrum disorder

“…These results, therefore, are generative models of the brain that provide Bayesian posterior estimates of the effective strength of synaptic connections among neuronal populations and modulatory or contextual effect of experimental manipulations (Friston, et al, 2003; Penny, et al, 2004). DCM also allows one to define models with different network properties, and then select the best model or the best family of models using Bayesian model comparison (Stephan, et al, 2009; Stephan, et al, 2010). …”

Probabilistic inference under time pressure leads to a cortical-to-subcortical shift in decision evidence integration

“…The optimal model represents a compensatory strategy model where participants were assumed to integrate all four cue dimensions in making their choices, whereas models 1 through 14 consisted of different variations of sub-optimal cue weight integration. To characterize decision strategies at the group level, we employed a Bayesian model selection procedure by submitting the log model evidences obtained from the variational Bayesian inference above (Rigoux et al, 2014;Stephan et al, 2009). This approach fits the hierarchical model by treating models as random effects that could vary across subjects, and estimates exceedance probabilities, which reflect the belief that a model, m, is more likely than any other model, given the marginalized likelihoods.…”

“…(3)-(5); Drugowitsch, 2013) for each model per participant. These log model evidences were then used to fit the hierarchical model (Rigoux et al, 2014;Stephan et al, 2009) to estimate the most likely strategy model employed in each experimental phase at the group level (Fig. 2C).…”

Dynamic causal modeling of load-dependent modulation of effective connectivity within the verbal working memory network

“…Model comparison was implemented using random-effects (RFX) Bayesian Model Selection (BMS) in DCM10 to compute exceedance and posterior probabilities at the group level (Stephan et al, 2009). The exceedance probability of a model denotes the probability that this model is more likely than any other in a given dataset.…”