Improving Clinical Translation of Machine Learning Approaches Through Clinician-Tailored Visual Displays of Black Box Algorithms: Development and Validation

Abstract

Background Despite the promise of machine learning (ML) to inform individualized medical care, the clinical utility of ML in medicine has been limited by the minimal interpretability and black box nature of these algorithms. Objective The study aimed to demonstrate a general and simple framework for generating clinically relevant and interpretable visualizations of black box predictions to aid in the clinical translation of ML. Methods To obtain improved transparency of ML, simplified models and visual displays can be generated using common methods from clinical practice such as decision trees and effect plots. We illustrated the approach based on postprocessing of ML predictions, in this case random forest predictions, and applied the method to data from the Left Ventricular (LV) Structural Predictors of Sudden Cardiac Death (SCD) Registry for individualized risk prediction of SCD, a leading cause of death. Results With the LV Structural Predictors of SCD Registry data, SCD risk predictions are obtained from a random forest algorithm that identifies the most important predictors, nonlinearities, and interactions among a large number of variables while naturally accounting for missing data. The black box predictions are postprocessed using classification and regression trees into a clinically relevant and interpretable visualization. The method also quantifies the relative importance of an individual or a combination of predictors. Several risk factors (heart failure hospitalization, cardiac magnetic resonance imaging indices, and serum concentration of systemic inflammation) can be clearly visualized as branch points of a decision tree to discriminate between low-, intermediate-, and high-risk patients. Conclusions Through a clinically important example, we illustrate a general and simple approach to increase the clinical translation of ML through clinician-tailored visual displays of results from black box algorithms. We illustrate this general model-agnostic framework by applying it to SCD risk prediction. Although we illustrate the methods using SCD prediction with random forest, the methods presented are applicable more broadly to improving the clinical translation of ML, regardless of the specific ML algorithm or clinical application. As any trained predictive model can be summarized in this manner to a prespecified level of precision, we encourage the use of simplified visual displays as an adjunct to the complex predictive model. Overall, this framework can allow clinicians to peek inside the black box and develop a deeper understanding of the most important features from a model to gain trust in the predictions and confidence in applying them to clinical care.

Development of Severe COVID-19 Adaptive Risk Predictor (SCARP), a Calculator to Predict Severe Disease or Death in Hospitalized Patients With COVID-19

“…Although random forests are notable for their impressive predictive ability, they have minimal interpretability. This often limits their adoption in clinical practice because of the lack of ability to communicate how the predictions were generated ( 22 ). To provide an interpretable visual display as a summary of the RF-SLAM predictions, we created summary regression trees for the 1-day and 7-day predictions as simplified visualizations of the algorithm.…”

Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review

“…However, to date, most of the research literature on AI in health care deals with the development, application, and evaluation of advanced analytic techniques and models [ 10 - 12 ], primarily within computer science, engineering, and medical informatics. The literature on the implementation of AI to improve existing clinical workflows is more fragmented and mostly based on nonempirical data from proof-of-concept studies [ 1 , 13 ] across multiple subject areas, such as data governance [ 14 ], ethics [ 15 ], accountability [ 3 ], interpretability [ 16 ], and regulation [ 17 ]. This means that there are uncertainties around factors that influence the implementation of AI in real-world health care setups [ 10 ] and that health care professionals lack guidance on how to implement AI in their daily practices [ 18 ].…”

A Knowledge Distillation Ensemble Framework for Predicting Short- and Long-Term Hospitalization Outcomes From Electronic Health Records Data

“…The pipeline comprises two stacked modules, each making predictions from a view of the patient's data: dynamic time-series and static features. The stacking of the pipeline's modules enables mimicking a clinician's approach to making prognostic decisions, by taking into account the interplay between the temporal Model interpretability has been extensively linked with clinical utility and trust in clinical risk prediction systems [56], where clinicians are generally reluctant to accept decisions guided by 'black-box' Machine Learning models [55], [67]. KD-OP enables the post-hoc interpretation of its predictions by extending the idea of using attention weights to interpret neural network outcomes [9].…”