Accuracy of Machine Learning Algorithms for the Diagnosis of Autism Spectrum Disorder: Systematic Review and Meta-Analysis of Brain Magnetic Resonance Imaging Studies

Abstract

Background In the recent years, machine learning algorithms have been more widely and increasingly applied in biomedical fields. In particular, their application has been drawing more attention in the field of psychiatry, for instance, as diagnostic tests/tools for autism spectrum disorder (ASD). However, given their complexity and potential clinical implications, there is an ongoing need for further research on their accuracy. Objective This study aimed to perform a systematic review and meta-analysis to summarize the available evidence for the accuracy of machine learning algorithms in diagnosing ASD. Methods The following databases were searched on November 28, 2018: MEDLINE, EMBASE, CINAHL Complete (with Open Dissertations), PsycINFO, and Institute of Electrical and Electronics Engineers Xplore Digital Library. Studies that used a machine learning algorithm partially or fully for distinguishing individuals with ASD from control subjects and provided accuracy measures were included in our analysis. The bivariate random effects model was applied to the pooled data in a meta-analysis. A subgroup analysis was used to investigate and resolve the source of heterogeneity between studies. True-positive, false-positive, false-negative, and true-negative values from individual studies were used to calculate the pooled sensitivity and specificity values, draw Summary Receiver Operating Characteristics curves, and obtain the area under the curve (AUC) and partial AUC (pAUC). Results A total of 43 studies were included for the final analysis, of which a meta-analysis was performed on 40 studies (53 samples with 12,128 participants). A structural magnetic resonance imaging (sMRI) subgroup meta-analysis (12 samples with 1776 participants) showed a sensitivity of 0.83 (95% CI 0.76-0.89), a specificity of 0.84 (95% CI 0.74-0.91), and AUC/pAUC of 0.90/0.83. A functional magnetic resonance imaging/deep neural network subgroup meta-analysis (5 samples with 1345 participants) showed a sensitivity of 0.69 (95% CI 0.62-0.75), specificity of 0.66 (95% CI 0.61-0.70), and AUC/pAUC of 0.71/0.67. Conclusions The accuracy of machine learning algorithms for diagnosis of ASD was considered acceptable by few accuracy measures only in cases of sMRI use; however, given the many limitations indicated in our study, further well-designed studies are warranted to extend the potential use of machine learning algorithms to clinical settings. Trial Registration PROSPERO CRD42018117779; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=117779

Divide and Conquer: Clustering Patients With Autism by Gene Expression Profiles Using Machine Learning Algorithms

“…Further, we currently do not have a su cient understanding of "what would work for whom", thereby limiting opportunities for maximizing outcomes for children and their families with economic rami cations for broader society. In this context, ML algorithms have been found to be useful in predicting diagnostic accuracy in ASD with neuroimaging data [58]. Further, one recent study used Gaussian Mixture Models and Hierarchical Agglomerative Clustering, which provide a statistical framework for learning latent cluster memberships to determine ASD subgroups with differentiated treatment responses [59].…”

Neuroimaging-Based Deep Learning in Autism Spectrum Disorder and Attention-Deficit/Hyperactivity Disorder

“…In a 2019 study by Moon et al [22], 40 machine learning algorithm studies for ASD published between 2007 and 2018 were systematically reviewed and meta-analysis was conducted. Among them, meta-analysis of 12 samples using structural MRI revealed that the integrated sensitivity was 0.83, specificity was 0.84, and area under the curve/partial area under the curve (AUC/pAUC) was 0.90/0.83.…”

“…fMRI measures neural activity using blood oxygenationlevel dependent (BOLD) differences. A 2019 meta-analysis study conducted on fMRI and machine learning algorithm studies on ASD published between 2007 and 2018 [22] found the integrated sensitivity to be 0.69, specificity to be 0.66, and AUC/pAUC to be 0.71.…”

Präzisionspsychiatrie und der Beitrag von Brain Imaging und anderen Biomarkern

“…Dieser MRT-basierte Ansatz erlaubte es dann, die Einteilung zwischen diesen beiden Störungen in einer separaten Patientenpopulation mit einer nahezu 70%igen Genauigkeit zu trennen [32]. Darüber hinaus finden ähnliche gelagerte Ansätze auch Anwendung im Bereich Autismus [33,34], Substanzabhängigkeiten [35] und Persönlichkeitsstörungen [36][37][38]. Diese Studien liefern somit klare Hinweise dafür, dass die in diagnostischen Studien erlernten neuroanatomischen Signaturen als klinische Entscheidungshilfen verwendet werden können.…”