SummaryData analysis workflows in many scientific domains have become increasingly complex and flexible. To assess the impact of this flexibility on functional magnetic resonance imaging (fMRI) results, the same dataset was independently analyzed by 70 teams, testing nine ex-ante hypotheses. The flexibility of analytic approaches is exemplified by the fact that no two teams chose identical workflows to analyze the data. This flexibility resulted in sizeable variation in hypothesis test results, even for teams whose statistical maps were highly correlated at intermediate stages of their analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Importantly, meta-analytic approaches that aggregated information across teams yielded significant consensus in activated regions across teams. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset. Our findings show that analytic flexibility can have substantial effects on scientific conclusions, and demonstrate factors related to variability in fMRI. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for multiple analyses of the same data. Potential approaches to mitigate issues related to analytical variability are discussed.
Limited statistical power due to small sample sizes is a problem in fMRI research. Most of the work to date has examined the impact of sample size on task-related activation, with less attention paid to the influence of sample size on brain-behavior correlations, especially in actual experimental fMRI data. We addressed this issue using two large data sets (a working memory task, N = 171, and a relational processing task, N = 865) and both univariate and multivariate approaches to voxelwise correlations. We created subsamples of different sizes and calculated correlations between task-related activity at each voxel and task performance. Across both data sets the magnitude of the brain-behavior correlations decreased and similarity across spatial maps increased with larger sample sizes. The multivariate technique identified more extensive correlated areas and more similarity across spatial maps, suggesting that a multivariate approach would provide a consistent advantage over univariate approaches in the stability of brain-behavior correlations. In addition, the multivariate analyses showed that a sample size of roughly 80 or more participants would be needed for stable estimates of correlation magnitude in these data sets. Importantly, a number of additional factors would likely influence the choice of sample size for assessing such correlations in any given experiment, including the cognitive task of interest and the amount of data collected per participant. Our results provide novel experimental evidence in two independent data sets that the sample size commonly used in fMRI studies of 20-30 participants is very unlikely to be sufficient for obtaining reproducible brain-behavior correlations, regardless of analytic approach.
Cortical atrophy and degraded axonal health have been shown to coincide during normal aging; however, few studies have examined these measures together. To lend insight into both the regional specificity and the relative timecourse of structural degradation of these tissue compartments across the adult lifespan, we analyzed gray matter (GM) morphometry (cortical thickness, surface area, volume) and estimates of white matter (WM) microstructure (fractional anisotropy, mean diffusivity) using traditional univariate and more robust multivariate techniques to examine age associations in 186 healthy adults aged 20–94 years old. Univariate analysis of each tissue type revealed that negative age associations were largest in frontal GM and WM tissue and weaker in temporal, cingulate, and occipital regions, representative of not only an anterior‐to‐posterior gradient, but also a medial‐to‐lateral gradient. Multivariate partial least squares correlation (PLSC) found the greatest covariance between GM and WM was driven by the relationship between WM metrics in the anterior corpus callosum and projections of the genu, anterior cingulum, and fornix; and with GM thickness in parietal and frontal regions. Surface area was far less susceptible to age effects and displayed less covariance with WM metrics, while regional volume covariance patterns largely mirrored those of cortical thickness. Results support a retrogenesis‐like model of aging, revealing a coupled relationship between frontal and parietal GM and the underlying WM, which evidence the most protracted development and the most vulnerability during healthy aging.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.