Single subject fMRI test–retest reliability metrics and confounding factors

Abstract

While the fMRI test-retest reliability has been mainly investigated from the point of view of group level studies, here we present analyses and results for single-subject test-retest reliability. One important aspect of group level reliability is that not only does it depend on between-session variance (test-retest), but also on between-subject variance. This has partly led to a debate regarding which reliability metric to use and how different sources of noise contribute to between-session variance. Focusing on single subject reliability allows considering between-session only. In this study, we measured test-retest reliability in four behavioural tasks (motor mapping, covert verb generation, overt word repetition, and a landmark identification task) to ensure generalisation of the results and at three levels of data processing (time-series correlation, t value variance, and overlap of thresholded maps) to understand how each step influences the other and how confounding factors influence reliability at each of these steps. The contributions of confounding factors (scanner noise, subject motion, and coregistration) were investigated using multiple regression and relative importance analyses at each step. Finally, to achieve a fuller picture of what constitutes a reliable task, we introduced a bootstrap technique of within- vs. between-subject variance. Our results show that (i) scanner noise and coregistration errors have little contribution to between-session variance (ii) subject motion (especially correlated with the stimuli) can have detrimental effects on reliability (iii) different tasks lead to different reliability results. This suggests that between-session variance in fMRI is mostly caused by the variability of underlying cognitive processes and motion correlated with the stimuli rather than technical limitations of data processing.

The pulvinar nucleus and antidepressant treatment: dynamic modeling of antidepressant response and remission with ultra-high field functional MRI

“…Several studies with only fair to moderate ICC values in pain paradigms were reported [53], while others find slightly better values [54]. Yet, spatial activation patterns [55] and noise [56] appear to be stable over measuring time-points. Since we detected significant differences in aMD patients only, we do not conclude that the pulvinar result is solely driven by heterogeneity potentially present between measurements or groups.…”

Resting-state test–retest reliability of a priori defined canonical networks over different preprocessing steps

“…Furthermore, it has been argued that global signal regression may be beneficial to deal with motion effects (Murphy et al, 2009; Power et al, 2012). In contrast, previous studies addressing the influence of global signal removal (Weissenbacher et al 2009; Chai et al 2012; Chen et al 2012) and those assessing test-retest reliability (Shehzad et al 2009; Gorgolewski et al 2013; Birn et al 2014) used less extensive motion regression protocols. Acknowledging new approaches based on automatically classifying and removing noise components have recently emerged (Behzadi et al 2007), we here focused on three steps commonly used in settings in which physiological noise recording is not available and data quality is not sufficient for reliable estimation of noise components in individual subjects.…”

“…Reliability of subjects (RoSO) and reliability of connections (RoCO) represent two fundamentally different views on reliability of resting-state measurements (Gorgolewski et al 2013). Conceptually, assessing the RoSO allows us to identify which combinations of processing steps that yield a reproducible relationship between subjects for each connection, while RoCO identify the combinations that yield the relationship between different connections in the same subject.…”

Improving the Test-Retest Reliability of Resting State fMRI by Removing the Impact of Sleep

“…Many factors contribute to the moderate reliability, including poor signal-to-noise ratio of the blood oxygenation level-dependent (BOLD) signal, excessive head motion, physiological noise, and so on. Previous work has found that test-retest reliability can be improved by removing volumes or subjects with excessive motion, and regressing out motion related artifacts (Schwarz and McGonigle, 2011; Guo et al, 2012; Zuo et al, 2012b; Gorgolewski et al, 2013; Yan et al, 2013; Du et al, 2015). Now we showed that the presence of drowsiness and sleep during scanning is another factor affecting rs-fMRI measures and their reliability.…”

“…Without external stimulation, one problem with resting state paradigm is the excessive head motion and associated scan artifacts (Van Dijk et al, 2012; Yan et al, 2013; Vanderwal et al, 2015). It has been showed that excessive head motion reduces the reliability of fMRI measures and excluding high motion subject or volumes, or regressing out motion related artifacts could improve the reliability of rs-fMRI measures (Schwarz and McGonigle, 2011; Guo et al, 2012; Zuo et al, 2012b; Gorgolewski et al, 2013; Yan et al, 2013; Du et al, 2015). …”