Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model

Abstract

We aim to produce predictive models that are not only accurate, but are also interpretable to human experts. Our models are decision lists, which consist of a series of if . . . then. . . statements (e.g., if high blood pressure, then stroke) that discretize a high-dimensional, multivariate feature space into a series of simple, readily interpretable decision statements. We introduce a generative model called Bayesian Rule Lists that yields a posterior distribution over possible decision lists. It employs a novel prior structure to encourage sparsity. Our experiments show that Bayesian Rule Lists has predictive accuracy on par with the current top algorithms for prediction in machine learning. Our method is motivated by recent developments in personalized medicine, and can be used to produce highly accurate and interpretable medical scoring systems. We demonstrate this by producing an alternative to the CHADS2 score, actively used in clinical practice for estimating the risk of stroke in patients that have atrial fibrillation. Our model is as interpretable as CHADS2, but more accurate. This is an electronic reprint of the original article published by the Institute of Mathematical Statistics in The Annals of Applied Statistics, 2015, Vol. 9, No. 3, 1350-1371. This reprint differs from the original in pagination and typographic detail. 1 2 LETHAM, RUDIN, MCCORMICK AND MADIGAN if male and adult then survival probability 21% (19%-23%) else if 3rd class then survival probability 44% (38%-51%) else if 1st class then survival probability 96% (92%-99%) else survival probability 88% (82%-94%)

How Does Predicate Invention Affect Human Comprehensibility?

“…Issues related to comprehensibility have been gaining more wide-spread attention recently in the study of classification models [4,6]. However, while these studies emphasise the need for comprehensibility, they do not offer a definitive test of the kind provided by our definition in Section 3.…”

Comparing sets of patterns with the Jaccard index

“…Other examples can be seen in Table 1. Patterns in any of these forms are sometimes referred to as decision rules or association rules, and when collected together can be called a decision list or rule list (Letham et al 2013). 1 1 For example, a decision tree might be "flattened" so as to no longer have roots or leaves, and it merely becomes an unordered set (a rule list) of unordered sets of attribute conditions (rules).…”

“…Fields such as classification can also find patterns by using decision forests (Breiman 2001a) or other classifiers (Han et al 2006). Patterns can be assessed for their usefulness (Geng & Hamilton 2006) and their interpretability (Letham et al 2013), or monitored for any changes in temporal scenarios (Baron et al 2003). While differing in methodology, approaches such as these agree on the importance of patterns and attest to the value of patterns for gaining knowledge.…”

“…Prediction accuracy is often used to measure a model's performance , Letham et al 2013, but is disconnected from any reliable assessments of pattern quality (Fletcher & Islam 2014, Islam et al 2003. A user should be aware of the specific goals they wish their model to achieve and how important the truthfulness (Kifer & Gehrke 2006) or interpretability of the patterns in the model are, and then use multiple measures to assess if their needs are met.…”

“…Assessing the patterns discovered by data mining algorithms is often overlooked in the current zeitgeist, instead favouring a dogged pursuit of higher prediction accuracy (Hand 2006, Wagstaff 2012. We argue that this is to the detriment of the discipline and provide a simple but powerful method for comparing sets of patterns, aiding a renewed focus in pattern assessment seen in recent years (Aodha et al 2014, Letham et al 2013.…”