Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI

Abstract

Brain magnetic resonance imaging (MRI) is useful for predicting the outcome of patients with acute ischemic stroke (AIS). Although deep learning (DL) using brain MRI with certain image biomarkers has shown satisfactory results in predicting poor outcomes, no study has assessed the usefulness of natural language processing (NLP)-based machine learning (ML) algorithms using brain MRI free-text reports of AIS patients. Therefore, we aimed to assess whether NLP-based ML algorithms using brain MRI text reports could predict poor outcomes in AIS patients. This study included only English text reports of brain MRIs examined during admission of AIS patients. Poor outcome was defined as a modified Rankin Scale score of 3–6, and the data were captured by trained nurses and physicians. We only included MRI text report of the first MRI scan during the admission. The text dataset was randomly divided into a training and test dataset with a 7:3 ratio. Text was vectorized to word, sentence, and document levels. In the word level approach, which did not consider the sequence of words, and the “bag-of-words” model was used to reflect the number of repetitions of text token. The “sent2vec” method was used in the sensation-level approach considering the sequence of words, and the word embedding was used in the document level approach. In addition to conventional ML algorithms, DL algorithms such as the convolutional neural network (CNN), long short-term memory, and multilayer perceptron were used to predict poor outcomes using 5-fold cross-validation and grid search techniques. The performance of each ML classifier was compared with the area under the receiver operating characteristic (AUROC) curve. Among 1840 subjects with AIS, 645 patients (35.1%) had a poor outcome 3 months after the stroke onset. Random forest was the best classifier (0.782 of AUROC) using a word-level approach. Overall, the document-level approach exhibited better performance than did the word- or sentence-level approaches. Among all the ML classifiers, the multi-CNN algorithm demonstrated the best classification performance (0.805), followed by the CNN (0.799) algorithm. When predicting future clinical outcomes using NLP-based ML of radiology free-text reports of brain MRI, DL algorithms showed superior performance over the other ML algorithms. In particular, the prediction of poor outcomes in document-level NLP DL was improved more by multi-CNN and CNN than by recurrent neural network-based algorithms. NLP-based DL algorithms can be used as an important digital marker for unstructured electronic health record data DL prediction.

Natural Language Processing Enhances Prediction of Functional Outcome After Acute Ischemic Stroke

“… 24 In this regard, magnetic resonance imaging studies are more sensitive than CT studies for detection of acute ischemia, 25 and magnetic resonance imaging reports seemed to be promising for predicting outcome after AIS. 26 Nevertheless, magnetic resonance imaging is not as widespread and readily available for emergency situations as CT.…”

“…In situations where model interpretability is given a high priority, simpler NLP approaches such as the BOW approach may be reasonable alternatives despite their lower predictive ability. 9 , 26 Furthermore, influential features identified from free text might be collected and used to develop new prognostic models. 10 …”

The Allure of Big Data to Improve Stroke Outcomes: Review of Current Literature

“…In one study, random forests performed best for identifying predictors of 30-day mortality after stroke [62••]. Models involving natural language processing of unstructured free-text magnetic resonance imaging reports have been successfully used to predict 90-day functional outcomes with great accuracy [63]. Machine learning methods have also been used to predict other stroke outcomes, including 30-day readmissions [60,64], home-time [50], motor recovery [65], and complications of stroke (e.g., pneumonia [66], and dysphagia [67]) and for developing a proxy measure of stroke severity [68].…”

Medical Specialty Recommendations by an Artificial Intelligence Chatbot on a Smartphone: Development and Deployment (Preprint)

“…The use of this data set helped develop an AI chatbot with which patients can interact easily and conveniently. Furthermore, this data set of Korean medical language may be useful for further NLP-based medical studies, including those on diagnosis, treatment, and prediction [31][32][33] because different languages have different features that can greatly influence the study of NLP.…”