Opening the black box of artificial intelligence for clinical decision support: A study predicting stroke outcome

Abstract

State-of-the-art machine learning (ML) artificial intelligence methods are increasingly leveraged in clinical predictive modeling to provide clinical decision support systems to physicians. Modern ML approaches such as artificial neural networks (ANNs) and tree boosting often perform better than more traditional methods like logistic regression. On the other hand, these modern methods yield a limited understanding of the resulting predictions. However, in the medical domain, understanding of applied models is essential, in particular, when informing clinical decision support. Thus, in recent years, interpretability methods for modern ML methods have emerged to potentially allow explainable predictions paired with high performance. To our knowledge, we present in this work the first explainability comparison of two modern ML methods, tree boosting and multilayer perceptrons (MLPs), to traditional logistic regression methods using a stroke outcome prediction paradigm. Here, we used clinical features to predict a dichotomized 90 days post-stroke modified Rankin Scale (mRS) score. For interpretability, we evaluated clinical features' importance with regard to predictions using deep Taylor decomposition for MLP, Shapley values for tree boosting and model coefficients for logistic regression. With regard to performance as measured by Area under the Curve (AUC) values on the test dataset, all models performed comparably: Logistic regression AUCs were 0.83, 0.83, 0.81 for three different regularization schemes; tree boosting AUC was 0.81; MLP AUC was 0.83. Importantly, the interpretability analysis demonstrated consistent results across models by rating age and stroke severity consecutively amongst the most important predictive features. For less important features, some differences were observed between the methods. Our analysis suggests that modern machine learning methods can provide explainability which is compatible with domain knowledge interpretation and traditional method rankings. Future work should focus on replication of these findings in other datasets and further testing of different explainability methods.

Predicting student satisfaction of emergency remote learning in higher education during COVID-19 using machine learning techniques

“…Nevertheless, deep learning techniques such as MLPR are regarded as "black box", which can limit further analysis on features' importance or the logic of the algorithm leading to prediction [84]. Therefore, with the good prediction accuracy of both MLPR and ENet observed in this study, ENet is considered as a better modeling choice.…”

Functional Outcome Prediction in Ischemic Stroke: A Comparison of Machine Learning Algorithms and Regression Models

“…For LASSO and the conventional logistic regression models, the magnitude of the standardized logistic regression coefficients was used rank order the predictors according to their importance ( 15 ). The absolute values of the importance metrics for all the predictors were scaled to unit norm, in order to ensure comparable rank ordering across all the investigated models ( 16 ).…”

Radiomics Detection of Pulmonary Hypertension via Texture-Based Assessments of Cardiac MRI: A Machine-Learning Model Comparison—Cardiac MRI Radiomics in Pulmonary Hypertension

“…The MLP model is a type of feedforward artificial neural network that has an input layer, an output layer, and multiple hidden layers. MLP helps in distinguishing data that is not linearly separable [30].…”

“…The MLP model is a type of feedforward artificial neural network that has an input layer, an output layer, and multiple hidden layers. MLP helps in distinguishing data that is not linearly separable [ 30 ]. The top five models for both primary and subgroup analysis were all fit to the full feature set, except for the fourth-best model which was fit using a high correlation filter.…”