Generalizability and reproducibility of functional connectivity in autism

Abstract

Background Autism is hypothesized to represent a disorder of brain connectivity, yet patterns of atypical functional connectivity show marked heterogeneity across individuals. Methods We used a large multi-site dataset comprised of a heterogeneous population of individuals with autism and typically developing individuals to compare a number of resting-state functional connectivity features of autism. These features were also tested in a single site sample that utilized a high-temporal resolution, long-duration resting-state acquisition technique. Results No one method of analysis provided reproducible results across research sites, combined samples, and the high-resolution dataset. Distinct categories of functional connectivity features that differed in autism such as homotopic, default network, salience network, long-range connections, and corticostriatal connectivity, did not align with differences in clinical and behavioral traits in individuals with autism. One method, lag-based functional connectivity, was not correlated to other methods in describing patterns of resting-state functional connectivity and their relationship to autism traits. Conclusion Overall, functional connectivity features predictive of autism demonstrated limited generalizability across sites, with consistent results only for large samples. Different types of functional connectivity features do not consistently predict different symptoms of autism. Rather, specific features that predict autism symptoms are distributed across feature types. Electronic supplementary material The online version of this article (10.1186/s13229-019-0273-5) contains supplementary material, which is available to authorized users.

Nonreplication of functional connectivity differences in autism spectrum disorder across multiple sites and denoising strategies

“…We should mention, however, that although there are different ways of dividing and grouping the data, these approaches mostly still fundamentally rest on the ability to accurately and reliably measure edge‐level differences in ASD (e.g., Yahata et al, ; see fig. 16a in King et al, ). For example, more complex statistical constructs that can be used to compare brain organization between groups (e.g., graph theoretic network measures; He et al, ; Rubinov & Sporns, ) fundamentally must build upon reliable and replicable measurement of connectomes.…”

“…Holiga et al () recently found replicable results regarding functional connectivity in ASD across four very large datasets that also included ABIDE data. However, another recent study by King et al () assessed a number of different measures of functional connectivity in ASD and found weak evidence of generalizability across sites. Other studies have used machine learning approaches to generalize to independently acquired datasets (e.g., Abraham et al, ; Yahata et al, ).…”

“…While various functional connectivity differences in ASD have been reported in previous studies, the overall lack of consensus is concerning. This issue has recently caught the attention of more researchers, and two additional recent studies also found nonreplicated group differences between sites (King et al, ; Nair et al, ). Importantly, a growing number of studies that now examine and demonstrate out‐of‐sample replication provide hope that such replicable signals may in fact exist (Holiga et al, ; Yahata et al, ).…”

“…(d) Additional experimental procedures can be employed to ensure more reliable estimates of an individual's connectome. For example, collecting more data from each individual participant can reduce measurement noise and ensure greater confidence in the results via within‐sample replication (Anderson, King, & Anderson, ; Byrge & Kennedy, ; Finn et al, ; King et al, ; Nee, ), prior to attempting across‐site replication.…”

“…ABIDE is a data sharing initiative wherein researchers across laboratories shared resting‐state data from TD and ASD participants for the flexible use by other researchers, “allow[ing] for replication, secondary analyses and discovery efforts” (Di Martino et al, ). This flexibility has allowed a proliferation of research on functional connectivity in ASD, and the data has been used in various ways, including considering each site separately (Hahamy, Behrmann, & Malach, ; Pua, Malpas, Bowden, & Seal, ) or using multisite aggregation (Abraham et al, ; Floris, Lai, Nath, Milham, & Di Martino, ; King et al, ). Both of these approaches are widely employed, and while greater statistical power can be achieved from aggregation, examining multiple individual sites can be used to evaluate replicability.…”