A clinically-translatable machine learning algorithm for the prediction of Alzheimer’s disease conversion: further evidence of its accuracy via a transfer learning approach

Abstract

Background:In a previous study, we developed a highly performant and clinically-translatable machine learning algorithm for a prediction of three-year conversion to Alzheimer’s disease (AD) in subjects with Mild Cognitive Impairment (MCI) and Pre-mild Cognitive Impairment. Further tests are necessary to demonstrate its accuracy when applied to subjects not used in the original training process. In this study, we aimed to provide preliminary evidence of this via a transfer learning approach.Methods:We initially employed the same baseline information (i.e. clinical and neuropsychological test scores, cardiovascular risk indexes, and a visual rating scale for brain atrophy) and the same machine learning technique (support vector machine with radial-basis function kernel) used in our previous study to retrain the algorithm to discriminate between participants with AD (n = 75) and normal cognition (n = 197). Then, the algorithm was applied to perform the original task of predicting the three-year conversion to AD in the sample of 61 MCI subjects that we used in the previous study.Results:Even after the retraining, the algorithm demonstrated a significant predictive performance in the MCI sample (AUC = 0.821, 95% CI bootstrap = 0.705–0.912, best balanced accuracy = 0.779, sensitivity = 0.852, specificity = 0.706).Conclusions:These results provide a first indirect evidence that our original algorithm can also perform relevant generalized predictions when applied to new MCI individuals. This motivates future efforts to bring the algorithm to sufficient levels of optimization and trustworthiness that will allow its application in both clinical and research settings.

A Novel Ensemble-Based Machine Learning Algorithm To Predict The Conversion From Mild Cognitive Impairment To Alzheimer’s Disease Using Socio-demographic Characteristics, Clinical Information And Neuropsychological Measures

“…Moreover, the first principal component of the three ADAS scores, which resulted in the most individually important predictor, demonstrated a test AUROC significantly lower than the one achieved by the entire algorithm. The results of our, as well as other previous studies, had already showed that machine learning algorithms can effectively be used to combine these individual pieces of information, providing a better identification of cAD among MCI subjects than what it would be possible using each of them singularly [13,14,19,20,46] . 19 Our study has some limitations that should be taken into account and that will be addressed in the future stages of our research.…”

“…Nevertheless, the fact that our new algorithm does not necessitate any magnetic resonance evaluation makes its use even more easily translatable in practice, and less expensive. Moreover, even though our former algorithm showed higher cross-validated performance (AUROC = 0.91, sensitivity = 86.7% and specificity = 87.4% at the best balanced accuracy) [19] ), a solid testing of its performance is still lacking and, at the moment, only a preliminary evidence via a transfer learning approach is available [20] . Instead, the protocol applied in the current study provides a better and sounder evaluation of the actual predictive performance of this new algorithm.…”

“…We previously presented another machine learning algorithm that performs a prediction of conversion to AD in MCI subjects [19,20] . However, the algorithm described here has distinct characteristics and can be considered at a more advanced stage of validation.…”

“…Future steps in our research will take into account this necessity, exploring the opportunity of making predictions at longer time periods and in earlier-stage subjects. 20…”

“…In the current age of big data and artificial intelligence technologies, considerable effort has been dedicated in developing machine learning algorithms that can predict the conversion to AD in subjects with MCI using different combinations of AD biomarkers with varying accuracy [7,[10][11][12][13][14][15][16][17][18] (see [19,20] for a recent review of the most performing algorithms presented in the scientific literature so far). Many studies focused on predicting the conversion of AD in MCI patients, combine different types of data including brain imaging, 2 CSF biomarkers, genotyping, demographic and clinical information, and cognitive performance.…”