Comparing fMRI inter-subject correlations between groups using permutation tests

Abstract

Inter-subject correlation (ISC) based analysis is a conceptually simple approach to analyze functional magnetic resonance imaging (fMRI) data acquired under naturalistic stimuli such as a movie. We describe and validate the statistical approaches for comparing ISCs between two groups of subjects implemented in the ISC toolbox, which is an open source software package for ISC-based analysis of fMRI data. The approaches are based on permutation tests. We validated the approaches using five different data sets from the ICBM functional reference battery tasks. First, we created five null datasets (one for each task) by dividing the subjects into two matched groups and assumed that no group difference exists. Second, based on one null dataset, we created datasets with simulated ISC differences of varying size between the two groups. Based on the experiments with these two types of data, we recommend the use of subject-wise permutations, instead of element-wise permutations. The tests based on subject-wise permutations led to correct false positive rates. We observed that the null-distributions should be voxel-specific and not based on pooling all voxels across the brain as is typical in fMRI. This was the case even if studentized permutation tests were used. Additionally, we experimented with an fMRI dataset acquired using a dance movie stimulus for comparison of a group of adult males on the autism spectrum to a matched typically developed group. The experiment confirmed the differences between voxel-based permutation tests and global model based permutation tests.

Exploration of human movement processing in autism spectrum disorder

“…Inter-subject Correlation Analysis. Inter-subject correlation (ISC) analysis was done following the methods described in previous research (Tohka, Pollick, Pajula, and Kauppi, 2019;Kauppi et al, 2014Kauppi et al, , 2010Pajula et al, 2012) using the ISC toolbox 2 . ISC was computed using an unpaired studentised between-group analysis (ASD-TD) for each movie separately and a paired between-movie analysis (Romantic-Classical) for each group separately.…”

“…Statistical Tests of ISC maps. ISCs between groups were compared with a voxel-null permutation test with subject-wise permutations (Tohka, Pollick, Pajula, and Kauppi, 2019). This test provides uncorrected p-values which were transformed into Z-values using p-to-Z transform.…”

Comparing Intersubject Correlation (ISC) Between Autism Spectrum Disorder and Typically Developed Groups to Better Understand Biological Motion Processing

“…Chen (2016) discusses two different permutation strategies that can be used to form a null distribution: subject-wise and element-wise permutations. Subject-wise permutation testing corresponds to the random swapping of subjects between the two groups before computing subject-pairwise ISCs, while element-wise permutation testing corresponds to exchanging the components of the correlation matrix (Tohka et al, 2018). Chen (2016) suggested that subject-wise permutations are more reliable for non-parametric statistical testing of two groups, because element-wise permutation leads to tests with excessively liberal hypothesis tests.…”

“…ISC is also used to highlight the different brain responses two different groups might have while experiencing the same stimulus (Hasson et al, 2009;Salmi et al, 2013). ISC is determined by calculating correlation coefficients between the fMRI time series of the subjects in corresponding brain locations and then averaging the coefficients (Tohka, Pollick, Pajula, & Kauppi, 2018).…”

“…Activation found in certain amounts of voxels over time dictates how large brain activation is in ISC maps. When using subject-wise permutation testing for ISC, two different null-models can be produced: either a null-model for each independent voxel (voxel-null model) can be used or it can be assumed that the same null model exists for all voxels (global null-model) (Tohka et al, 2018).…”