Mind the gap: Performance metric evaluation in brain‐age prediction

Lange, Ann-Marie G. de; Anatürk, Melis; Rokicki, Jaroslav; Han, Laura K.M.; Franke, Katja; Alnæs, Dag; Ebmeier, Klaus P.; Draganski, Bogdan; Kaufmann, Tobias; Westlye, Lars T.; Hahn, Tim; Cole, James R.

doi:10.1002/hbm.25837

Cited by 85 publications

(97 citation statements)

References 94 publications

(249 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each algorithm was trained using grid search to find the best parameters that give the highest accuracy. The performance of each algorithm was quantified by the Pearson’s correlation coefficient (r) and mean absolute error (MAE) between predicted brain age and chronological age [ 6 ]. We also reported weighted MAE for comparison between studies with different sample age ranges.…”

Section: Methodsmentioning

confidence: 99%

“…The biological age of the brain (“brain age”) is estimated typically by applying a machine learning approach to magnetic resonance imaging (MRI) data to predict chronological age. The difference between an individual’s predicted brain age and actual chronological age is referred to here as brain-predicted age difference (brainPAD) [ 2 , 3 ], which is also known as brain age gap [ 4 , 5 ] or brain age delta [ 6 ]. This metric reflects the deviation from expected age trajectories and is often used to index brain health [ 1 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Brain Age Prediction: A Comparison between Machine Learning Models Using Brain Morphometric Data

Han

Kim

Lee

et al. 2022

Sensors

View full text Add to dashboard Cite

Brain structural morphology varies over the aging trajectory, and the prediction of a person’s age using brain morphological features can help the detection of an abnormal aging process. Neuroimaging-based brain age is widely used to quantify an individual’s brain health as deviation from a normative brain aging trajectory. Machine learning approaches are expanding the potential for accurate brain age prediction but are challenging due to the great variety of machine learning algorithms. Here, we aimed to compare the performance of the machine learning models used to estimate brain age using brain morphological measures derived from structural magnetic resonance imaging scans. We evaluated 27 machine learning models, applied to three independent datasets from the Human Connectome Project (HCP, n = 1113, age range 22–37), the Cambridge Centre for Ageing and Neuroscience (Cam-CAN, n = 601, age range 18–88), and the Information eXtraction from Images (IXI, n = 567, age range 19–86). Performance was assessed within each sample using cross-validation and an unseen test set. The models achieved mean absolute errors of 2.75–3.12, 7.08–10.50, and 8.04–9.86 years, as well as Pearson’s correlation coefficients of 0.11–0.42, 0.64–0.85, and 0.63–0.79 between predicted brain age and chronological age for the HCP, Cam-CAN, and IXI samples, respectively. We found a substantial difference in performance between models trained on the same data type, indicating that the choice of model yields considerable variation in brain-predicted age. Furthermore, in three datasets, regularized linear regression algorithms achieved similar performance to nonlinear and ensemble algorithms. Our results suggest that regularized linear algorithms are as effective as nonlinear and ensemble algorithms for brain age prediction, while significantly reducing computational costs. Our findings can serve as a starting point and quantitative reference for future efforts at improving brain age prediction using machine learning models applied to brain morphometric data.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Brain Age Prediction: A Comparison between Machine Learning Models Using Brain Morphometric Data

Han

Kim

Lee

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…We observed lower wMAEs in BASE-II when compared to UK Biobank, suggesting lower model accuracies in BASE-II. Notably, wMAE has been reported to vary as a function of age range, as it does not account for the underlying age distribution (de Lange et al, 2022). Therefore, we additionally matched UK Biobank to BASE-II participants by chronological age and sex using R package MatchIt (Ho et al, 2011), and recalculated prediction accuracies for the resulting UK Biobank subset (for details see section "Comparison of Age Prediction Accuracies in BASE-II and UK Biobank" in Supplementary Material A).…”

Section: Prediction Accuraciesmentioning

confidence: 99%

Linking Brain Age Gap to Mental and Physical Health in the Berlin Aging Study II

Jawinski

Markett

Drewelies

et al. 2022

Front. Aging Neurosci.

View full text Add to dashboard Cite

From a biological perspective, humans differ in the speed they age, and this may manifest in both mental and physical health disparities. The discrepancy between an individual’s biological and chronological age of the brain (“brain age gap”) can be assessed by applying machine learning techniques to Magnetic Resonance Imaging (MRI) data. Here, we examined the links between brain age gap and a broad range of cognitive, affective, socioeconomic, lifestyle, and physical health variables in up to 335 adults of the Berlin Aging Study II. Brain age gap was assessed using a validated prediction model that we previously trained on MRI scans of 32,634 UK Biobank individuals. Our statistical analyses revealed overall stronger evidence for a link between higher brain age gap and less favorable health characteristics than expected under the null hypothesis of no effect, with 80% of the tested associations showing hypothesis-consistent effect directions and 23% reaching nominal significance. The most compelling support was observed for a cluster covering both cognitive performance variables (episodic memory, working memory, fluid intelligence, digit symbol substitution test) and socioeconomic variables (years of education and household income). Furthermore, we observed higher brain age gap to be associated with heavy episodic drinking, higher blood pressure, and higher blood glucose. In sum, our results point toward multifaceted links between brain age gap and human health. Understanding differences in biological brain aging may therefore have broad implications for future informed interventions to preserve mental and physical health in old age.

show abstract

“…Most people, who showed advanced age in one measure, also did so in other measures. Furthermore, as argued elsewhere (de Lange et al, 2022), performance metrics such as MAE are sample-specific (i.e., they cannot easily be compared across studies) and should be considered in combination with other metrics such as correlation coefficients. For example and similar to our study, large increases in MAE have been reported despite acceptable correlations (r > 0.5) between chronological and predicted age, especially in situations where the age range of the testing sample was very narrow and different to that of the training sample (de Lange et al, 2022).…”

Section: Discussionmentioning

confidence: 99%

Associations between methylation age and brain age in late adolescence

Sanders

Baltramonaityte

Donohoe

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent research suggests that biological age, based on DNA methylation or neuroimaging measures, may predict health traits in adulthood more accurately than chronological age. However, whether these findings apply to earlier stages in life is unknown. We therefore aimed to characterise the performance of and interdependence between measures of biological age during adolescence, leveraging longitudinal data from a subsample of young adolescents from the population-based ALSPAC cohort (n=386). We derived four methylation age measures in late adolescence (17-19 years) and a measure of brain age derived from structural neuroimaging data (18-24 years). We then examined associations between these measures of biological age, and their relationship with five measures of physical, cognitive and mental health (8-18 years). Brain age was largely independent of different measures of methylation age, even after accounting for age, cell type composition, array and study (beta range: -0.60 to 0.17, all p>0.05). Smoking and BMI predicted three measures of advanced methylation age (beta range: -0.39 to 0.52, all p<0.05), but not brain age. Depressive symptoms and cognitive ability were unrelated to all measures of biological age. Our findings emphasize the variability of and independence between these methylation- and brain-based measures of age in adolescents. They also highlight the importance of tracking the mosaic of ageing in younger populations.

show abstract

Mind the gap: Performance metric evaluation in brain‐age prediction

Cited by 85 publications

References 94 publications

Brain Age Prediction: A Comparison between Machine Learning Models Using Brain Morphometric Data

Brain Age Prediction: A Comparison between Machine Learning Models Using Brain Morphometric Data

Linking Brain Age Gap to Mental and Physical Health in the Berlin Aging Study II

Associations between methylation age and brain age in late adolescence

Contact Info

Product

Resources

About