Meta-matching as a simple framework to translate phenotypic predictive models from big to small data

Abstract

There is significant interest in using brain imaging data to predict non-brain-imaging phenotypes in individual participants. However, most prediction studies are underpowered, relying on less than a few hundred participants, leading to low reliability and inflated prediction performance. Yet, small sample sizes are unavoidable when studying clinical populations or addressing focused neuroscience questions. Here, we propose a simple framework -"meta-matching" -to translate predictive models from large-scale datasets to new unseen non-brain-imaging phenotypes in boutique studies. The key observation is that many large-scale datasets collect a wide range inter-correlated phenotypic measures.Therefore, a unique phenotype from a boutique study likely correlates with (but is not the same as) some phenotypes in some large-scale datasets. Meta-matching exploits these correlations to boost prediction in the boutique study. We applied meta-matching to the problem of predicting non-brain-imaging phenotypes using resting-state functional connectivity (RSFC). Using the UK Biobank (N = 36,848), we demonstrated that metamatching can boost the prediction of new phenotypes in small independent datasets by 100% to 400% in many scenarios. When considering relative prediction performance, metamatching significantly improved phenotypic prediction even in samples with 10 participants.When considering absolute prediction performance, meta-matching significantly improved phenotypic prediction when there were least 50 participants. With a growing number of largescale population-level datasets collecting an increasing number of phenotypic measures, our results represent a lower bound on the potential of meta-matching to elevate small-scale boutique studies..

Is resting state fMRI better than individual characteristics at predicting cognition?

“…However, most of these studies have utilized the same datasets, primarily the widely used Human Connectome Project (HCP) database [66], with a medium sample size of fewer than 1500 participants. Cognitive prediction studies that use the UK Biobank database have a lot more participants than HCP, but as far as the authors know, only a few studies with large fMRI datasets have been done so far [22], [56], [57]. This makes extrapolating the findings of low sample size studies to larger sample sizes challenging [5].…”

Exploring the latent structure of behavior using the Human Connectome Project’s data

“…As it is not needed for the behavioral variables to be the same to still arrive at similar summary dimensions, datasets with very different behavioral assays can be combined to train predictive models, if one is willing to go one step up in granularity and predict these broader behavioral dimensions. A similar approach to this was already demonstrated by He and colleagues 46 , who leveraged the correlations between behavioral variables to improve the predictive accuracy of predictive models using the functional connectome. There are several avenues to build on these results.…”

Bias in machine learning models can be significantly mitigated by careful training: Evidence from neuroimaging studies

“…Even for balanced datasets, trained models may still be biased due to unobserved confounders, e.g., severity of the disease or genetic factors. Recent studies have therefore argued for training models on extensive multisource neuroimaging datasets ( 24 ). Our results agree with such motivations—large-scale cohorts of multisource data can enable training robust and unbiased models and also enable thorough evaluation.…”