Data Analysis With Shapley Values For Automatic Subject Selection in Alzheimer's Disease Data Sets Using Interpretable Machine Learning

Abstract

Background: The prediction of whether Mild Cognitive Impaired (MCI) subjects will prospectively develop Alzheimer's Disease (AD) is important for the recruitment and monitoring of subjects for therapy studies. Machine Learning (ML) is suitable to improve early AD prediction. The etiology of AD is heterogeneous, which leads to noisy data sets. Additional noise is introduced by multicentric study designs and varying acquisition protocols. This article examines whether an automatic and fair data valuation method based on Shapley values can identify subjects with noisy data. Methods: An ML-workow was developed and trained for a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. The validation was executed for an independent ADNI test data set and for the Australian Imaging, Biomarker and Lifestyle Flagship Study of Ageing (AIBL) cohort. The workow included volumetric Magnetic Resonance Imaging (MRI) feature extraction, subject sample selection using data Shapley, Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) for model training and Kernel SHapley Additive exPlanations (SHAP) values for model interpretation. This model interpretation enables clinically relevant explanation of individual predictions. Results: The XGBoost models which excluded 116 of the 467 subjects from the training data set based on their Logistic Regression (LR) data Shapley values outperformed the models which were trained on the entire training data set and which reached a mean classification accuracy of 58.54 % by 14.13 % (8.27 percentage points) on the independent ADNI test data set. The XGBoost models, which were trained on the entire training data set reached a mean accuracy of 60.35 % for the AIBL data set. An improvement of 24.86 % (15.00 percentage points) could be reached for the XGBoost models if those 72 subjects with the smallest RF data Shapley values were excluded from the training data set. Conclusion: The data Shapley method was able to improve the classification accuracies for the test data sets. Noisy data was associated with the number of ApoEϵ4 alleles and volumetric MRI measurements. Kernel SHAP showed that the black-box models learned biologically plausible associations.

Application of Explainable Artificial Intelligence in Alzheimer's Disease Classification: A Systematic Review

“…The research article used a numeric data type dataset as input for subjects of different categories like HC, MCI, or AD selected from the Alzheimers Disease Neuroimaging Initiative dataset (ADNI). Bloch et al [116] state that the diverse causes of AD can lead to inconsistencies in disease patterns, protocols used for acquiring scans, and preprocessing errors of MRI scans resulting in improper ML classification. This study investigates whether selecting the most informative participants from the ADNI and Australian Imaging Biomarker and Lifestyle (AIBL) cohorts can enhance ML classification using an automatic and fair data valuation method based on XAI techniques.…”

“…The random forest model has been used in numerous research to classify AD ( [115,116,144,147]) using SHAP to depict the explanations using violin, force, and summary plots (see Fig. 15, Fig.…”

“…The ML or DL learning algorithms predict dominant classes more accurately than classes with inadequate samples. Most studies had limited AD samples compared to HC or MCI cohorts [158,116,118,144,150,154,126]. Only a balanced dataset can draw meaningful insights.…”

A Machine Learning Approach for Early Diagnosis of Cognitive Impairment Using Population-Based Data

“…To address the issue of black-box predictions in ML, we applied an explainer model, Shapley Additive Explanations (SHAP) to data generated by the ensemble ML model to understand how the algorithm make its prediction. This method has been previously described [5,42,43] and was applied to studies on dementia and cognitive impairment. More details on SHAP explainer model can be found in Supplementary Material 2.…”

“…F early diagnosis will help identify individuals at risk and allow the adoption of preventive measures to mitigate this disease [2,3]. The prodromal stages of dementia is characterized by mild cognitive impairment (MCI) [4], i.e., individuals with noticeable decline in mental abilities not interfering with everyday activities [5]. Previous studies suggest that individuals with MCI are at a higher risk of dementia than cognitively healthy individuals [6,7].…”