Effect Size and Power in fMRI Group Analysis

Abstract

Multi-subject functional magnetic resonance imaging (fMRI) analysis is often concerned with determining whether there exists a significant population-wide 'activation' in a comparison between two or more conditions. Typically this is assessed by testing the average value of a contrast of parameter estimates (COPE) against zero in a general linear model (GLM) analysis. In this work we investigate several aspects of this type of analysis. First, we study the e↵ects of sample size on the sensitivity and reliability of the group analysis, allowing us to evaluate the ability of small sampled studies to e↵ectively capture population-level e↵ects of interest. Second, we assess the di↵erence in sensitivity and reliability when using volumetric or surface based data. Third, we investigate potential biases in estimating e↵ect sizes as a function of sample size. To perform this analysis we utilize the task-based fMRI data from the 500-subject release from the Human Connectome Project (HCP). We treat the complete collection of subjects (N = 491) as our population of interest, and perform a single-subject analysis on each subject in the population. We investigate the ability to recover population level e↵ects using a subset of the population and standard analytical techniques. Our study shows that sample sizes of 40 are generally able to detect regions with high e↵ect sizes (Cohen's d > 0.8), while sample sizes closer to 80 are required to reliably recover regions with medium e↵ect sizes (0.5 < d < 0.8). We find little di↵erence in results when using volumetric or surface based data with respect to standard mass-univariate group analysis. Finally, we conclude that special care is needed when estimating e↵ect sizes, particularly for small sample sizes.

Investigating GABA concentrations measured with macromolecule suppressed and unsuppressed MEGA‐PRESS MR spectroscopy and their relationship with BOLD responses in the occipital cortex

“…These results also highlight the problems inherent in studying associations between variables derived from low‐sensitivity methods, such as fMRI and MRS, in terms of the sample sizes required. For fMRI research, recent work has helped demonstrate the limits of making inferences from small samples, suggesting that larger groups (>100) than have normally been used are necessary for robust and replicable analyses . Equivalent studies for MRS are not yet available, but the meta‐analysis reported here helps illustrate the problem in this context.…”

“…For fMRI research, recent work has helped demonstrate the limits of making inferences from small samples, suggesting that larger groups (>100) than have normally been used are necessary for robust and replicable analyses. 30,31 Equivalent studies for MRS are not yet available, but the meta-analysis reported here helps illustrate the problem in this context. Adding the current results to five previous studies produced overall evidence for a negative association between GABA estimates and BOLD amplitudes (total sample size = 100).…”

Power, positive predictive value, and sample size calculations for random field theory-based fMRI inference

“…However, it is well known that eect size estimates resulting from the thresholding of mass-univariate statistical parametric maps exhibit biases (e.g., Vul et al, 2009;Poldrack et al, 2017). To correct our eect size estimate for this bias, we capitalized on recent results by Geuter et al (2018), which are depicted in the lower panel of Figure 4A. Specically, using task-related fMRI data from the Human Connectome Figure 4A, this eect size bias is most severe for small data subsets and decreases with increasing data subset size.…”

“…In the recent debate on the reproducibility of research ndings in the life sciences, the statistical practices of fMRI research have once again taken centre stage in the community discourse (e.g., Eklund et al, 2016;Mumford et al, 2016;Poldrack et al, 2017;Eklund et al, 2019;Flandin and Friston, 2019). Here, a particular emphasis has been on statistical power and its relation to typical sample sizes in fMRI group studies (Button et al, 2013;Guo et al, 2014;Szucs and Ioannidis, 2016;Cremers et al, 2017;Geuter et al, 2018;Turner et al, 2018). In task-related fMRI, statistical power is broadly dened as the probability of detecting cortical activation, if this activation is indeed present.…”

Shared striatal activity in decisions to satisfy curiosity and hunger at the risk of electric shocks

“…The sample sizes of the fMRI experiments were determined to be comparable to or larger than other neuroimaging studies of motivational biases 105 and curiosity-based effects 5,6 . Also, such a sample size should be sufficient to reliably detect medium effect sizes in task-based fMRI analysis 106 .…”