Extracting seizure frequency from epilepsy clinic notes: a machine reading approach to natural language processing

Abstract

Objective Seizure frequency and seizure freedom are among the most important outcome measures for patients with epilepsy. In this study, we aimed to automatically extract this clinical information from unstructured text in clinical notes. If successful, this could improve clinical decision-making in epilepsy patients and allow for rapid, large-scale retrospective research. Materials and Methods We developed a finetuning pipeline for pretrained neural models to classify patients as being seizure-free and to extract text containing their seizure frequency and date of last seizure from clinical notes. We annotated 1000 notes for use as training and testing data and determined how well 3 pretrained neural models, BERT, RoBERTa, and Bio_ClinicalBERT, could identify and extract the desired information after finetuning. Results The finetuned models (BERTFT, Bio_ClinicalBERTFT, and RoBERTaFT) achieved near-human performance when classifying patients as seizure free, with BERTFT and Bio_ClinicalBERTFT achieving accuracy scores over 80%. All 3 models also achieved human performance when extracting seizure frequency and date of last seizure, with overall F1 scores over 0.80. The best combination of models was Bio_ClinicalBERTFT for classification, and RoBERTaFT for text extraction. Most of the gains in performance due to finetuning required roughly 70 annotated notes. Discussion and Conclusion Our novel machine reading approach to extracting important clinical outcomes performed at or near human performance on several tasks. This approach opens new possibilities to support clinical practice and conduct large-scale retrospective clinical research. Future studies can use our finetuning pipeline with minimal training annotations to answer new clinical questions.

Transforming epilepsy research: A systematic review on natural language processing applications

“…NLP was employed in six studies for extracting epilepsy‐specific variables from EHRs and scientific articles 12,32–36 . The variables of interest identified from the reports included patients' demographics, epilepsy etiology and diagnoses, brain imaging results, laterality information, seizure semiology and occurrence, and medications.…”

“…27 3.2.2 | Structured information retrieval NLP was employed in six studies for extracting epilepsyspecific variables from EHRs and scientific articles. 12,[32][33][34][35][36] The variables of interest identified from the reports included patients' demographics, epilepsy etiology and diagnoses, brain imaging results, laterality information, seizure semiology and occurrence, and medications. Automated extraction of the information above speeds up the research process by enabling rapid patient cohort identification from EHRs and scientific papers.…”

Clinical signatures of genetic epilepsy precede diagnosis in electronic medical records of 32,000 individuals

“…There has been recent work demonstrating that more advanced NLP exploiting deep learning can extract detailed information such as seizure frequency from the EMR of individuals with epilepsy more accurately. 35 Integrating such tools could provide a more accurate, deeper picture of an individual's clinical landscape.…”

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

“…[79] Word embeddings are typically learned from a large corpus in an unsupervised fashion and used as the input layer to a neural network. Common corpora within the selected articles included clinical notes [53,57,63,[80][81][82][83][84] as well as external sources such as biomedical publications [56,61,62,85,86] and Wikipedia articles [51,58,[87][88][89][90] (Table S9). Word2vec, [91] Global Vectors (GloVE), [92] and Bidirectional Encoder Representations from Transformers (BERT) [93][94][95][96] were the most frequently used methods for training embeddings (Table S10).…”