Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations

Abstract

Cluster-extent based thresholding is currently the most popular method for multiple comparisons correction of statistical maps in neuroimaging studies, due to its high sensitivity to weak and diffuse signals. However, cluster-extent based thresholding provides low spatial specificity; researchers can only infer that there is signal somewhere within a significant cluster and cannot make inferences about the statistical significance of specific locations within the cluster. This poses a particular problem when one uses a liberal cluster-defining primary threshold (i.e., higher p-values), which often produces large clusters spanning multiple anatomical regions. In such cases, it is impossible to reliably infer which anatomical regions show true effects. From a survey of 814 functional magnetic resonance imaging (fMRI) studies published in 2010 and 2011, we show that the use of liberal primary thresholds (e.g., p < .01) is endemic, and that the largest determinant of the primary threshold level is the default option in the software used. We illustrate the problems with liberal primary thresholds using an fMRI dataset from our laboratory (N = 33), and present simulations demonstrating the detrimental effects of liberal primary thresholds on false positives, localization, and interpretation of fMRI findings. To avoid these pitfalls, we recommend several analysis and reporting procedures, including 1) setting primary p < .001 as a default lower limit; 2) using more stringent primary thresholds or voxel-wise correction methods for highly powered studies; and 3) adopting reporting practices that make the level of spatial precision transparent to readers. We also suggest alternative and supplementary analysis methods.

Structural plasticity of the social brain: Differential change after socio-affective and cognitive mental training

“…The findings were corrected for multiple comparisons using random field theory for nonisotropic images (105). Given previous discussions in the fMRI community on the impact of cluster-forming thresholds on overall FWE levels and interpretability (106,107), statistical results were corrected for multiple comparisons by means of random field theory using both typically used (108-111) cluster-defining thresholds for 20-mm FWHM smoothed surface-based 2D thickness data [where higher smoothing kernels relate to more readily fulfilled assumptions of Gaussian random field theory (106,112)] and a more conservative cluster-forming threshold recently recommended for the analysis of 3D voxel-based functional data smoothed with smaller, isotropic kernels. We therefore superimposed significant findings on the basis of a cluster-forming threshold of P = 0.025 with a more stringent cluster-forming threshold of P = 0.001.…”

Domain-general involvement of the posterior frontolateral cortex in time-based resource-sharing in working memory: An fMRI study

“…Indeed, cluster-extent based thresholding detects statistically significant clusters on the basis of the number of contiguous voxels whose statistic values exceed a threshold; this does not mean that every voxel in the cluster displays the significant effect (Woo et al, 2014). As a result, one cannot be certain that the overlapping clusters of voxels identified in the conjunction analysis using cluster-extent thresholding entail any voxels that showed significant effects in their respective initial contrast.…”

Not on my team: Medial prefrontal cortex responses to ingroup fusion and unfair monetary divisions

“…This contrast revealed a cluster that extended over a large portion of the mPFC (Figure 2), including both the DMPFC (areas 8 and 9) and VMPFC (area 32), in the same region as recently identified within a meta‐analysis of UG studies (Feng et al., 2015; Gabay et al., 2014). This large cluster survived whole brain cluster correction ( p <  0.001 uncorrected voxel‐wise; p  <   0.05 FWE cluster‐correction) at the level recommended in recent analyses of corrections for multiple comparisons (Eklund et al., 2016; Woo et al., 2014). Notably, we did not find any responses in other regions implicated in the UG, such as the Insula, showing a group by fairness interaction effect.…”

“…Firstly, for the main, first contrast above we set a voxelwise threshold of p  < 0.001 uncorrected, but then corrected at the cluster threshold of p  < 0.05 FWE (Eklund, Nichols, & Knutsson, 2016; Woo, Krishnan, & Wager, 2014). To then specify the location of our results with greater precision, we used masks of areas 8, 9, 24 and both the dorsal and ventral portions of area 32 from the study of Neubert and colleagues (Neubert, Mars, Sallet, & Rushworth, 2015).…”