Automated information extraction from free-text EEG reports

Abstract

In this study we have developed a supervised learning to automatically detect with high accuracy EEG reports that describe seizures and epileptiform discharges. We manually labeled 3,277 documents as describing one or more seizures vs no seizures, and as describing epileptiform discharges vs no epileptiform discharges. We then used Naïve Bayes to develop a system able to automatically classify EEG reports into these categories. Our system consisted of normalization techniques, extraction of key sentences, and automated feature selection using cross validation. As candidate features we used key words and special word patterns called elastic word sequences (EWS). Final feature selection was accomplished via sequential backward selection. We used cross validation to predict out of sample performance. Our automated feature selection procedure resulted in a classifier with 38 features for seizure detection, and 23 features for epileptiform discharge detection. The average [95% CI] area under the receiver operating curve was 99.05 [98.79, 99.32]% for detecting reports with seizures, and 96.15 [92.31, 100.00]% for detecting reports with epileptiform discharges. The methodology described herein greatly reduces the manual labor involved in identifying large cohorts of patients for retrospective neurophysiological studies of patients with epilepsy.

Machine learning applications in epilepsy

“…Connolly et al examined progress notes for pediatric patients with focal, generalized, or otherwise unspecified epilepsies, analyzing frequencies of word strings using an SVM classifier; after training on 90 notes of each category from one institution, the algorithm demonstrated better‐than‐chance classification of progress notes from a different institution, with improved performance if the classifier was trained using notes from two institutions before being tested on notes from a third . Biswal et al, in contrast, trained a Naïve Bayes classifier on 3277 EEG reports labeled with the presence or absence of seizures or epileptiform discharges, which then achieved an AUC of 99.05% for detecting reports with seizures and 96.15% for those with epileptiform discharges on a testing set of 39 695 reports . Building on this approach, Goodwin and Harabagiu combined automated indexing of EEG reports with neural network‐generated ”fingerprints” of the associated recordings to create a searchable database, allowing for identification of patient cohorts within the database with such queries as, “History of seizures and EEG with TIRDA without sharps, spikes, or electrographic seizures”; a panel of expert reviewers found that the addition of EEG “fingerprint” indices provided more relevant results and captured pertinent records not recovered by searches of EEG reports alone .…”

“…59 Biswal et al, in contrast, trained a Naïve Bayes classifier on 3277 EEG reports labeled with the presence or absence of seizures or epileptiform discharges, which then achieved an AUC of 99.05% for detecting reports with seizures and 96.15% for those with epileptiform discharges on a testing set of 39 695 reports. 60 Building on this approach, Goodwin and Harabagiu combined automated indexing of EEG reports with neural network-generated "fingerprints" of the associated recordings to create a searchable database, allowing for identification of patient cohorts within the database with such queries as, "History of seizures and EEG with TIRDA without sharps, spikes, or electrographic seizures" 61 ; a panel of expert reviewers found that the addition of EEG "fingerprint" indices provided more relevant results and captured pertinent records not recovered by searches of EEG reports alone. 61 The broad range of data sources examined in these studies highlights the utility of machine learning techniques in leveraging underutilized clinical data for patient-and populationlevel characterization of epilepsy.…”

Can antiepileptic drug efficacy be studied from electronic health records? A review of current approaches

“…Biswal et al achieved an AUC of 0.99 for detecting reports with seizures and AUC of 0.96 for epileptiform discharges, but a limitation of their study is that they did not differentiate focal from generalized findings. [36]…”

“…Biswal et al achieved an AUC of 0.99 for detecting reports with seizures and AUC of 0.96 for epileptiform discharges, but a limitation of their study is that they did not differentiate focal from generalized findings. [36] Bao reported 94% accuracy in interictal EEG diagnosis from raw EEG signal. However, this does not improve on the accuracy obtained from structured fields and unstructured reports, and increases the computational resources required for analysis as well as the administrative cost to obtain the raw EEG signal data.…”

Can antiepileptic efficacy and epilepsy variables be studied from electronic health records? A review of current approaches

“…In addition to direct analysis of raw EEG signal, EEG abnormalities can be captured by means of structured fields and unstructured reports. Biswal et al achieved an AUC of 0.99 for detecting reports with seizures and AUC of 0.96 for epileptiform discharges, but they did not differentiate focal from generalized findings [36].…”