ADHD-200 Global Competition: diagnosing ADHD using personal characteristic data can outperform resting state fMRI measurements

Abstract

Neuroimaging-based diagnostics could potentially assist clinicians to make more accurate diagnoses resulting in faster, more effective treatment. We participated in the 2011 ADHD-200 Global Competition which involved analyzing a large dataset of 973 participants including Attention deficit hyperactivity disorder (ADHD) patients and healthy controls. Each participant's data included a resting state functional magnetic resonance imaging (fMRI) scan as well as personal characteristic and diagnostic data. The goal was to learn a machine learning classifier that used a participant's resting state fMRI scan to diagnose (classify) that individual into one of three categories: healthy control, ADHD combined (ADHD-C) type, or ADHD inattentive (ADHD-I) type. We used participants' personal characteristic data (site of data collection, age, gender, handedness, performance IQ, verbal IQ, and full scale IQ), without any fMRI data, as input to a logistic classifier to generate diagnostic predictions. Surprisingly, this approach achieved the highest diagnostic accuracy (62.52%) as well as the highest score (124 of 195) of any of the 21 teams participating in the competition. These results demonstrate the importance of accounting for differences in age, gender, and other personal characteristics in imaging diagnostics research. We discuss further implications of these results for fMRI-based diagnosis as well as fMRI-based clinical research. We also document our tests with a variety of imaging-based diagnostic methods, none of which performed as well as the logistic classifier using only personal characteristic data.

Computational neuroimaging strategies for single patient predictions

“…This has been variably successful, as indicated by the diverse outcomes from fMRI-based classification competitions. In these competitions, the accuracies of neuroimaging-based diagnoses have ranged from poor (e.g., attention deficit hyperactivity disorder; Brown et al, 2012) to excellent (e.g., schizophrenia; Silva et al, 2014). More importantly, however, the attempt to replace or augment traditional clinical diagnostics by applying machine learning techniques to neuroimaging data is a strategy of limited long-term clinical utility.…”

Cross-validation and hypothesis testing in neuroimaging: An irenic comment on the exchange between Friston and Lindquist et al.

“…The approach of Eloyan et al (2012) predicted the test data diagnoses most accurately—among teams who used the imaging data. Surprisingly, however, Brown et al (2012) attained somewhat better predictive accuracy without using the images at all. My colleagues and I (Reiss et al, 2015) have shown how CV-based testing can be modified to assess whether image data adds predictive value beyond that offered by non-imaging predictors.…”

Developmental Resting State Functional Connectivity for Clinicians

“…A multi-site data sharing effort termed ADHD-200 [58], has recently resulted in publications with very large samples. For example, ADHD brains (n=757) exhibited altered RSFC between default network and ventral attention networks [59].…”