Bootstrapping promotes the RSFC‐behavior associations: An application of individual cognitive traits prediction

Wei, Lijiang; Jing, Bin; Li, Haiyun

doi:10.1002/hbm.24947

Cited by 19 publications

(17 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, in these implementations, the maximum number of feature selection iterations would be 10. Given that with the 100 bootstraps, we get almost 50% of features occurring at least once, and that another study by Wei et al also recommend ~100 bootstraps [25], bagging may provide an improvement on identifying noisy features compared to 10-fold cross validation, as well as giving more insight into feature variation. Furthermore, the greater the fraction of subjects from the training set that are included in the feature selection step, in the case of 10-fold CV it would include 90% of the subjects, the less likely the features are to vary across folds.…”

Section: Discussionmentioning

confidence: 89%

“…The CV models cited above, which have been applied successfully out-of-sample, have in essence used a bagging approach. Bagging has been applied to fMRI data before to determine brain state [23], and for brain parcellation (to define an atlas) [24], and has also been shown to boost within-sample performance of resting state FC based brain-behavior models [25]. Here we apply a bagged predictive modeling approach, using connectome predictive modeling (CPM) as the model framework.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bootstrap aggregating improves the generalizability of Connectome Predictive Modelling

O’Connor

Lake

Scheinost

et al. 2020

Preprint

View full text Add to dashboard Cite

It is a long-standing goal of neuroimaging to produce reliable generalized models of brain behavior relationships. More recently data driven predicative models have become popular. Overfitting is a common problem with statistical models, which impedes model generalization. Cross validation (CV) is often used to give more balanced estimates of performance. However, CV does not provide guidance on how best to apply the models generated out-of-sample. As a solution, this study proposes an ensemble learning method, in this case bootstrap aggregating, or bagging, encompassing both model parameter estimation and feature selection. Here we investigate the use of bagging when generating predictive models of fluid intelligence (fIQ) using functional connectivity (FC). We take advantage of two large openly available datasets, the Human Connectome Project (HCP), and the Philadelphia Neurodevelopmental Cohort (PNC). We generate bagged and non-bagged models of fIQ in the HCP. Over various test-train splits, these models are evaluated in sample, on left out HCP data, and out-of-sample, on PNC data. We find that in sample, a non-bagged model performs best, however out-of-sample the bagged models perform best. We also find that feature selection can vary substantially within-sample. A more considered approach to feature selection, alongside data driven predictive modeling, is needed to improve cross sample performance of FC based brain behavior models.

show abstract

Section: Discussionmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Bootstrap aggregating improves the generalizability of Connectome Predictive Modelling

O’Connor

Lake

Scheinost

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…There was, however, little information about the measurement's validity for the populations under study. 3 studies cited references deemed adequate (Ferguson et al 2017;Wei et al 2020;Yang et al 2019), whereas partial references were cited in 2 studies (Hilger et al 2020;Lin et al 2020).…”

Section: Resultsmentioning

confidence: 99%

“…for detailed ratings Ferguson et al 2017;Finn et al 2015;Gao et al 2019;Greene et al 2018;He et al 2020;Jiang et al 2020a;Kashyap et al 2019;Kong et al 2019;Li et al 2018;Li et al 2019;Lin et al 2020;Powell et al 2017;Wei et al 2020;Wu et al 2020;Yang et al 2019;Yoo et al 2019;Zhang et al 2019. …”

unclassified

On the prediction of human intelligence from neuroimaging: A systematic review of methods and reporting

Vieira¹,

Pamplona

Fachinello

et al. 2021

Preprint

View full text Add to dashboard Cite

Human intelligence is one of the main objects of study in cognitive neuroscience. Reviews and meta-analyses have proved to be fundamental to establish and cement neuroscientific theories on intelligence. The prediction of intelligence using in vivo neuroimaging data and machine learning has become a widely accepted and replicated result. Here, we present a systematic review of this growing area of research, based on studies that employ structural, functional, and/or diffusion MRI to predict human intelligence in cognitively normal subjects using machine-learning. We performed a systematic assessment of methodological and reporting quality, using the PROBAST and TRIPOD assessment forms and 30 studies identified through a systematic search. We observed that fMRI is the most employed modality, resting-state functional connectivity (RSFC) is the most studied predictor, and the Human Connectome Project is the most employed dataset. A meta-analysis revealed a significant difference between the performance obtained in the prediction of general and fluid intelligence from fMRI data, confirming that the quality of measurement moderates this association. The expected performance of studies predicting general intelligence from fMRI was estimated to be r = 0.42 (CI95% = [0.35, 0.50]) while for studies predicting fluid intelligence obtained from a single test, expected performance was estimated as r = 0.15 (CI95% = [0.13, 0.17]). We further enumerate some virtues and pitfalls we identified in the methods for the assessment of intelligence and machine learning. The lack of treatment of confounder variables, including kinship, and small sample sizes were two common occurrences in the literature which increased risk of bias. Reporting quality was fair across studies, although reporting of results and discussion could be vastly improved. We conclude that the current literature on the prediction of intelligence from neuroimaging data is reaching maturity. Performance has been reliably demonstrated, although extending findings to new populations is imperative. Current results could be used by future works to foment new theories on the biological basis of intelligence differences.

show abstract

“…Feature selection was performed to reduce the dimensionality of FC data. In this study, we used a bootstrap-based feature selection method (Wei et al, 2020). The goal of this method is to identify stable features that can be consistently identified among resampled subsets.…”

Section: Feature Selection: Resampling and Correlation Analysismentioning

confidence: 99%

Predicting the Degree of Distracted Driving Based on fNIRS Functional Connectivity: A Pilot Study

Ogihara

Tanioka

Hiroyasu

et al. 2022

Preprint

View full text Add to dashboard Cite

Distracted driving is one of the main causes of traffic accidents. By predicting the attentional state of drivers, it is possible to prevent distractions and promote safe driving. In this study, we developed a model that could predict the degree of distracted driving based on brain activity. Changes in oxyhemoglobin concentrations were measured in drivers while driving a real car using functional near-infrared spectroscopy (fNIRS). A regression model was constructed for each participant using functional connectivity as an explanatory variable and break reaction time to random beeps while driving as an objective variable. As a result, we were able to construct a prediction model for 11 of the 12 participants. Furthermore, the regression model with the highest prediction accuracy for each participant was analyzed to gain a better understanding of the neural basis of distracted driving. The 11 models were classified into five clusters by hierarchical clustering based on their functional connectivity edges used in each cluster. The results showed that the combinations of the dorsal attention network (DAN) sensory motor network (SMN) and DAN ventral attention network (VAN) connections were common in all clusters and that these networks were essential to predict the degree of distraction in complex multitask driving. They also confirmed the existence of multiple types of prediction models with different within- and between-network connectivity patterns. These results indicate that it is possible to predict the degree of distracted driving based on the brain activity of the driver during actual driving. These results are expected to contribute to the development of safe driving systems and elucidation of the neural basis of distracted driving.

show abstract

Bootstrapping promotes the RSFC‐behavior associations: An application of individual cognitive traits prediction

Cited by 19 publications

References 81 publications

Bootstrap aggregating improves the generalizability of Connectome Predictive Modelling

Bootstrap aggregating improves the generalizability of Connectome Predictive Modelling

On the prediction of human intelligence from neuroimaging: A systematic review of methods and reporting

Predicting the Degree of Distracted Driving Based on fNIRS Functional Connectivity: A Pilot Study

Contact Info

Product

Resources

About