A simple permutation-based test of intermodal correspondence

Abstract

Many key findings in neuroimaging studies involve similarities between brain maps. While several statistical procedures have been proposed to test correspondence between maps, there remains no consensus on the correct framing of a null hypothesis or a suitable testing approach. We propose a simple yet powerful permutation-based testing procedure for assessing similarities between two modalities using subject-level data. Our proposed method is similar to traditional permutation procedures in that it involves randomly permuting subjects to generate a null distribution. However, it differs from other recently proposed methods that have involved spherical rotations of the cortical surface or spatial autocorrelation-preserving ''surrogate'' maps of the brain, which depend on strong and potentially unrealistic statistical assumptions. To address these issues, we first demonstrate in simulated data that our method is conservative in terms of type I error and has high power. Next, we illustrate that our method performs well for assessing intermodal relationships from multimodal magnetic resonance imaging data from the Philadelphia Neurodevelopmental Cohort. The proposed test rejects the null hypothesis for modalities for which there is known interdependence in structure (cortical thickness and sulcal depth) but not in cases where an association would not be predicted biologically (cortical thickness and activation on the n-back working memory task). In contrast to previous methods, our approach does not depend on strong statistical assumptions other than the independence of subjects. Notably, our method is the most flexible for analyzing intermodal correspondence within subregions of the brain and has the greatest potential to be used for generalizable statistical inference.

Spatially-enhanced clusterwise inference for testing and localizing intermodal correspondence

“…It is also informative to consider a setting in which we do not expect to see strong intermodal associations. Thus, we consider correspondence tests between cortical thickness and n-back task activation, as previous work has not suggested we would expect either global or local correspondence between these modalities [9]. Cortical thickness is quantified as the minimum distance between pial and white matter surfaces.…”

“…Methodological developments in this area have included statistical estimation of individual-level maps measuring intermodal coupling [1,2,3,4,5] and hypothesis tests for establishing statistical significance of intermodal associations [6,7,8,9].…”

“…In our current work, we propose a novel hypothesis testing method that involves a combination of approaches from Weinstein et al [9] (SPICE) and recent work by Park and Fiecas [10] that explicitly models spatial autocorrelation in cluster-level inference. Using Park and Fiecas [10]'s spatial modeling framework, our method leverages spatial information through cluster-level inference after adjusting for differences in the spatial autocorrelation structure of each modality.…”

“…Using Park and Fiecas [10]'s spatial modeling framework, our method leverages spatial information through cluster-level inference after adjusting for differences in the spatial autocorrelation structure of each modality. Using a permutation framework similar to our previous work [9], our new method controls type I error rates at the nominal level and exhibits high statistical power for both global testing and localization of intermodal correspondence, as assessed in simulations and analyses of structure-function coupling in data from the Philadelphia Neurodevelopmental Cohort.…”

Brain connectivity at rest predicts individual differences in normative activity during movie watching

“…This procedure is repeated until a desired number (here, 10,000) of permutations have been performed, and the final permutation P value reflects the number of null permutations for which the resulting Spearman correlation is greater than the observed correlation divided by the total number of permutations. Although recent work has identified that this approach relies upon often unrealistic statistical assumptions (Weinstein et al, 2020), we note that no alternative method is compatible with our effect maps which are undefined at the individual subject level.…”