Development of Expert-Level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation

Abstract

Background and Objectives:Seizures and other seizure-like patterns of brain activity can harm the brain and contribute to in-hospital death, particularly when prolonged. However, experts qualified to interpret electroencephalography (EEG) data are scarce. Prior attempts to automate this task have been limited by small or inadequately labeled samples and have not convincingly demonstrated generalizable expert-level performance. There exists a critical unmet need for an automated method to classify seizures and other seizure-like events with expert-level reliability. This study was conducted to develop and validate a computer algorithm that matches the reliability and accuracy of experts in identifying seizures and seizure-like events, known as “ictal-interictal-injury-continuum” (IIIC) patterns on EEG, including seizures (SZ), lateralized and generalized periodic discharges (LPD, GPD), and lateralized and generalized rhythmic delta activity (LRDA, GRDA), and in differentiating these patterns from non-IIIC patterns.Methods:We used 6,095 scalp EEGs from 2,711 patients with and without IIIC events to train a deep neural network,SPaRCNet, to perform IIIC event classification. Independent training and test datasets were generated from 50,697 EEG segments, independently annotated by 20 fellowship-trained neurophysiologists. We assessed whetherSPaRCNetperforms at or above the sensitivity, specificity, precision, and calibration of fellowship-trained neurophysiologists for identifying IIIC events. Statistical performance was assessed via the calibration index, and by the percentage of experts whose operating points were below the model’s receiver operating characteristic curves (ROC) and precision recall curves (PRC) for the 6 pattern classes.Results:SPaRCNetmatches or exceeds most experts in classifying IIIC events based on both calibration and discrimination metrics. For SZ, LPD, GPD, LRDA, GRDA, and “Other” classes,SPaRCNetexceeds the following percentages of 20 experts – ROC: 45%, 20%, 50%, 75%, 55%, 40%; PRC: 50%, 35%, 50%, 90%, 70%, 45%; and calibration: 95%, 100%, 95%, 100%, 100%, 80%, respectively.Discussion:SPaRCNetis the first algorithm to match expert performance in detecting seizures and other seizure-like events in a representative sample of EEGs. With further development,SPaRCNetmay thus be a valuable tool for expedited review of EEGs.

Putting the “Big” in Big Data

“…1 Next, they used those expert classifications as a training set to teach SPaRCNet how to distinguish the different patterns, using the majority vote of the experts as the benchmark. 2 The massive scale in these 2 works provides a level of rigor and interpretation that has never been possible before.…”

“…In this issue of Neurology ® , 2 companion papers address multiple hurdles to deploying such automated algorithms with a new detector, Seizures, Periodic and Rhythmic pattern Continuum (SPaRCNet). 1,2 SPaRCNet is a deep convolutional neural network, a deep learning tool that was originally developed to characterize images. In this case, it learns how to discriminate between different EEG patterns by training on markings made by expert clinicians, that is, it learns to do with the neural network what the clinicians do by eye.…”

“…The comparison between human reviewers provided a metric for success-the algorithm simply had to be as good (or uncertain) as a typical expert. In the study described in the second paper, Jing et al 2 trained and benchmarked SPaRCNet. The authors found that in an apples-to-apples comparison, SPaRCnet indeed performed as well as the human reviewers at differentiating IIIC patterns and had better prediction accuracy (known as calibration).…”

“…A near-term application of this work could be deployment alongside clinicians using automated clustering and feature visualization. Jing et al 2 explored this use-case finding that clinicians using their tools could review the model results in just a few minutes for a long EEG record. While full deployment will take more fine-tuning and has several challenges, the validation of this tool is already complete, far exceeding what is typically required for approval by governmental agencies.…”

“…A second concern is that black box algorithms provide no insight into their decisions, making it difficult to handle clinical “gray areas.” To address this concern, Jing et al provided a 2-dimensional projection of the IIIC classes (Figure 3). 2 The map looks like a starfish: a central hub (“other” patterns that include normal activity) that branches out to the 5 IIIC patterns. They show with examples how moving farther out on each arm makes the pattern more clearly identified, but the patterns meld as one moves in closer to the center; it is easy to see how reviewers could disagree about ambiguous patterns.…”