Predicting individual traits from models of brain dynamics accurately and reliably using the Fisher kernel

Abstract

How specifically brain activity unfolds across time, namely the nature of brain dynamics, can sometimes be more predictive of behavioural and cognitive subject traits than both brain structure and summary measures of brain activity that average across time. Brain dynamics can be described by models of varying complexity, but what is the best way to use these models of brain dynamics for characterising subject differences and predicting individual traits is unclear. While most studies aiming at predicting subjects’ traits focus on having accurate predictions, for many practical applications, it is critical for the predictions not just to be accurate but also reliable. Kernel methods are a robust and computationally efficient way of expressing differences in brain dynamics between subjects for the sake of predicting individual traits, such as clinical or psychological variables. Using the Hidden Markov model (HMM) as a model of brain dynamics, we here propose the use of the Fisher kernel, a mathematically principled approach to predict phenotypes from subject-specific HMMs. The Fisher kernel is computed in a way that preserves the mathematical structure of the HMM. This results in benefits in terms of both accuracy and reliability when compared with kernels that ignore the structure of the underlying model of brain dynamics.

Stacking models of brain dynamics improves prediction of subject traits in fMRI

“…For each base-level prediction, an HMM was run on a temporally concatenated fMRI timeseries of all subjects to obtain a group-level HMM (Vidaurre et al, 2017, 2021) ( Figure 1a ). We then used the Fisher kernel method, a mathematically principled approach to predicting target variables from an HMM (Ahrends et al, 2023; Jaakkola & Haussler, 1998) ( Figure 1b ). We compared each subject’s timeseries to the group-level HMM in the HMM’s parameter space, assuming that similar subjects would induce similar parameter gradients in this space.…”

“…When fixing the hyperparameters, we chose δ = 10, the default setting in the HMM-MAR toolbox, and K = 6, commonly used values in recent literature (Alonso & Vidaurre, 2023; Quinn et al, 2018; Vidaurre et al, 2018). The choice of states represents a compromise between having a higher number of states, which can increase the chances of certain states being present in only a subset of subjects (Ahrends et al, 2023), and a lower number of states, which can increase the chance of assigning entire sessions to a single state (i.e., model stasis) (Ahrends et al, 2022). Nevertheless, the specific choice of hyperparameters is less critical to our study, as our focus when fixing the hyperparameter is to investigate run-to-run variability which could be done with an alternative configuration.…”

“…To predict subject traits from the HMM, we used the Fisher kernel method (Ahrends et al, 2023; Jaakkola et al, 2000; Jaakkola & Haussler, 1998), which is a mathematically principled approach to predict from generative probabilistic models. To do this, we first estimate subject-specific HMM parameter sets (i.e., initial state probabilities, transition probability matrices, and brain state means and covariance) using dual estimation (Vidaurre et al, 2021).…”

“…The Fisher kernel is constructed from these subject-specific HMM parameter sets and can be used with a kernel-based classification or prediction method, here KRR, while respecting the structure of the HMM. Specifically, the Fisher kernel is constructed upon the Riemannian manifold formed by the parameters that define the HMM (Ahrends et al, 2023; Jaakkola et al, 2000; Jaakkola & Haussler, 1998). This method therefore provides an approach, given a certain choice of hyperparameters, to predict subject traits from time-varying patterns of activity and FC.…”

Predicting subject traits from brain spectral signatures: an application to brain ageing

“…Since we cannot straightforwardly predict from the raw data, an intermediate representation is typically used for prediction. In the case of fMRI, this is often a simple description of functional connectivity (Rosenberg et al (2016)) or some model of brain dynamics (Liegeois et al (2019); Vidaurre et al (2021); Ahrends et al (2023)). Here, we instead focus on EEG, a considerably less costly technique.…”