Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization With Medical Applications

Abstract

Medical applications challenge today's text categorization techniques by demanding both high accuracy and ease-of-interpretation. Although deep learning has provided a leap ahead in accuracy, this leap comes at the sacrifice of interpretability. To address this accuracy-interpretability challenge, we here introduce, for the first time, a text categorization approach that leverages the recently introduced Tsetlin Machine. In all brevity, we represent the terms of a text as propositional variables. From these, we capture categories using simple propositional formulae, such as: if "rash" and "reaction" and "penicillin" then Allergy. The Tsetlin Machine learns these formulae from a labelled text, utilizing conjunctive clauses to represent the particular facets of each category. Indeed, even the absence of terms (negated features) can be used for categorization purposes. Our empirical comparison with Naïve Bayes, decision trees, linear support vector machines (SVMs), random forest, long shortterm memory (LSTM) neural networks, and other techniques, is quite conclusive. The Tsetlin Machine either performs on par with or outperforms all of the evaluated methods on both the 20 Newsgroups and IMDb datasets, as well as on a non-public clinical dataset. On average, the Tsetlin Machine delivers the best recall and precision scores across the datasets. Finally, our GPU implementation of the Tsetlin Machine executes 5 to 15 times faster than the CPU implementation, depending on the dataset. We thus believe that our novel approach can have a significant impact on a wide range of text analysis applications, forming a promising starting point for deeper natural language understanding with the Tsetlin Machine.

Predicting cause of death from free-text health summaries: development of an interpretable machine learning tool

“…For a machine learning tool to be useful, especially in the medical domain, it is essential for the user to be able to interpret its output. As such, it is common for studies about clinical decision support or prediction modelling to include a discussion of interpretability (29)(30)(31). However, as Lipton points out, "interpretability is not a monolithic concept" (32)it includes distinct yet intersecting ideas such as comprehension, transparency and trust.…”

Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability

“…Despite being rule-based, TMs have obtained competitive performance in terms of accuracy, memory footprint, and inference speed on diverse benchmarks, including image classification, regression, natural language understanding, and game-playing. Employing a team of TA [28], a TM learns a linear combination of conjunctive clauses in propositional logic, producing decision rules similar to the branches in a decision tree (e.g., if X satisfies condition A and not condition B then Y = 1) [23].…”

“…The TM can further be used in convolution, providing competitive performance on MNIST, Fashion-MNIST, and Kuzushiji-MNIST, in comparison with CNNs, K-Nearest Neighbour (KNN), SVMs, RF, Gradient Boosting, BinaryConnect, Logistic Circuits and ResNet [29]. The TM has also achieved promising results in text classification by using the conjunctive clauses to capture textual patterns [23]. Further, hyper-parameter search can be simplified with multi-granular clauses, eliminating the pattern specificity parameter [25].…”

Optimized deep belief network and entropy-based hybrid bounding model for incremental text categorization

“…The aim of text classification is to categorize the large-sized documents into different categories on the basis of its contents (Mohammad et al , 2018). There exist numerous methods for performing text-related tasks such as profiling users, sentiment analysis and identification of spams, which is considered as a supervised learning issue and is addressed with text classifiers (Berge et al , 2019). The text classifier contains different sub-processes in which some of them are more flexible such that it can be adapted for solving the issues of supervised learning whereas other classifiers are specially developed for addressing a specific task using expensive processes such as syntactic analysis and lemmatization (Tellez et al , 2018).…”