Use of Clinical Pathway Simulation and Machine Learning to Identify Key Levers for Maximizing the Benefit of Intravenous Thrombolysis in Acute Stroke

Abstract

Background: Expert opinion is that about 20% of emergency stroke patients should receive thrombolysis. Currently, 11% to 12% of patients in England and Wales receive thrombolysis, ranging from 2% to 24% between hospitals. The aim of this study was to assess how much variation is due to differences in local patient populations, and how much is due to differences in clinical decision-making and stroke pathway performance, while estimating a realistic target thrombolysis use. Methods: Anonymised data for 246 676 emergency stroke admissions to 132 acute hospitals in England and Wales between 2016 and 2018 was obtained from the Sentinel Stroke National Audit Programme data. We used machine learning to learn decisions on who to give thrombolysis to at each hospital. We used clinical pathway simulation to model effects of changing pathway performance. Qualitative research was used to assess clinician attitudes to these methods. Three changes were modeled: (1) arrival-to-treatment in 30 minutes, (2) proportion of patients with determined stroke onset times set to at least the national upper quartile, (3) thrombolysis decisions made based on majority vote of a benchmark set of hospitals. Results: Of the modeled changes, any single change was predicted to increase national thrombolysis use from 11.6% to between 12.3% to 14.5% (clinical decision-making having the most effect). Combined, these changes would be expected to increase thrombolysis to 18.3%, but there would still be significant variation between hospitals depending on local patient population. Clinicians engaged well with the modeling, but those from hospitals with lower thrombolysis use were most cautious about the methods. Conclusions: Machine learning and clinical pathway simulation may be applied at scale to national stroke audit data, allowing extended use and analysis of audit data. Stroke thrombolysis rates of at least 18% look achievable in England and Wales, but each hospital should have its own target.

What would other emergency stroke teams do? Using explainable machine learning to understand variation in thrombolysis practice

“…The model predicted hospital thrombolysis use at each hospital with very good accuracy (r-squared = 0.977, figure 1). This maintains the high accuracy of our previously published model with overall accuracy of 84.3% (83.2% sensitivity and specificity could be achieved simultaneously), and mean ROC AUC of 0.906 14 .…”

“…We have built on our previous work to predict thrombolysis use from patient level data, by creating an explainable machine learning model which maintains the high accuracy that we previously achieved (85%) 14 . The model was well-calibrated, with the average predicted probability of thrombolysis for any group of patients matching the proportion of those patients who did receive thrombolysis.…”

“…In our previous work we established that we could predict the use of thrombolysis in patients arriving within 4 hours of known stroke onset with 84.3% accuracy 14 . We could then ask the question "What if this patient attended another hospital -would they likely be given thrombolysis?"…”

Using machine learning and clinical registry data to uncover variation in clinical decision making

“…The NIHSS is a 15 item neurological scale that is used by clinicians as a measure of stroke severity. It ranges from 0 to 42 and may be used to classify a stroke as mild (NIHSS 1-4), moderate (NIHSS 5-14), severe (NIHSS [15][16][17][18][19][20][21][22][23][24] or very severe (NIHSS > 25) [16]. Also present in the data is a patient's modified Rankin Scale (mRS) score before stroke, which is a measure of pre-stroke disability ranging from 0 to 5, with 0 corresponding to no prior disability and 5 meaning severely disabled.…”

“…In the example presented we used random forest algorithms due to their robustness, and previous work [24] has shown that, for this specific example, RF performs comparably to a neural network, and is superior to a logistic regression. However, the methods we have presented are more general and can be used alongside any ML algorithm.…”