Applying text-mining to clinical notes: The identification of patient characteristics from Electronic Health Records (EHRs)

Abstract

Background Clinical notes contain information on critical patient characteristics, which, if overlooked, could escalate the risk of adverse events as well as miscommunication between the healthcare professional and the patient. This study investigates the feasibility of employing text-mining to extract patient characteristics from Electronic Health Records (EHRs) and compares the effectiveness of text-mining against human intelligence for identifying four patient characteristics: language barrier, living alone, cognitive frailty and non-adherence. Methods A manual “golden” standard was created from 1,120 patient files (878 patients) that had unplanned hospital readmissions. Each patient was categorized under one of the four characteristics with supporting free text extracted from their EHRs. For simple terminology, a rule-based (RB) SQL query was used, and for complex terms, Named Entity Recognition (NER) models were used. Model performance was compared to the manual standard. The primary outcomes were recall, specificity, precision, negative predictive value (NPV) and F1-score. Results Performance of each patient characteristic was evaluated using a separate train/test dataset and an additional evaluation dataset was used for the NER models. Within the train/test set, the ‘language barrier’ RB query achieved a recall of 0.99 (specificity of 0.96). The ‘living alone’ NER model showed a recall of 0.86 (specificity of 0.94) on the train/test set and a recall of 0.81 (specificity of 1.00) on the validation set. In that same order, the ‘cognitive frailty’ model yielded a recall of 0.59 (specificity 0.76) on the train/test set and a recall of 0.73 (specificity 0.96) on the validation set. The NER model for non-adherence showed a recall of 0.75 (specificity of 0.99) on the train/test set, and a recall of 0.90 (specificity of 0.99) on the validation set. The models showed the tendency to overestimate the presence of patient characteristics such as identifying a family member’s language barrier as the patient’s. Conclusion This study successfully demonstrated the feasibility of applying text-mining to identify patient characteristics from EHRs. Also, it seems for more complex terminology, NER models outperform the rule-based option. Future work involves refining these models for broader application and terminology in clinical setting.