Interrater reliability of EEG-video monitoring

Abstract

Objective: The diagnosis of psychogenic nonepileptic seizures (PNES) can be challenging. In the absence of a gold standard to verify the reliability of the diagnosis by EEG-video, we sought to assess the interrater reliability of the diagnosis using EEG-video recordings.Methods: Patient samples consisted of 22 unselected consecutive patients who underwent EEGvideo monitoring and had at least an episode recorded. Other test results and histories were not provided because the goal was to assess the reliability of the EEG-video. Data were sent to 22 reviewers, who were board-certified neurologists and practicing epileptologists at epilepsy centers. Choices were 1) PNES, 2) epilepsy, and 3) nonepileptic but not psychogenic ("physiologic") events. Interrater agreement was measured using a coefficient for each diagnostic category. We used generalized coefficients, which measure the overall level of between-method agreement beyond that which can be ascribed to chance. We also report category-specific values. Results: Conclusions:Interrater reliability for the diagnosis of psychogenic nonepileptic seizures by EEGvideo monitoring was only moderate. Although this may be related to limitations of the study (diagnosis based on EEG-video alone, artificial nature of the forced choice paradigm, single episode), it highlights the difficulties and subjective components inherent to this diagnosis.

Interictal High Frequency Oscillations Detected with Simultaneous Magnetoencephalography and Electroencephalography as Biomarker of Pediatric Epilepsy

“…Although this approach is often regarded as the gold standard, it presents serious limitations since it has poor inter-reviewer reliability 77,78 , and is not applicable to large MEG and EEG datasets with high number of sensors. Crucial to the application of HFOs in clinical practice is the development of algorithms that detect the HFOs automatically from scalp recordings reducing the need for human input.…”

Computer assisted interpretation of the human EEG

“…An important finding in (Halford et al, 2011) was the relatively large inter-rater variance (also reported by others, e.g. (Azuma et al, 2003;Benbadis et al, 2009)), which points to an even greater need to use a single, commonly accepted dataset for benchmarking purposes.…”

“…Although this approach still holds the highest specificity and sensitivity for finding abnormalities, it is time consuming and requires extensive training. Apart from this, visual analysis is susceptible to reviewer bias (Haut et al, 2002;Benbadis et al, 2009;Gerber et al, 2008;Azuma et al, 2003). Automated quantification techniques reduce the time required for reviewing and improve consistency in reporting.…”

“…The la er part can lead to a high degree of variability and inconsistency between reviewers (Haut et al, 2002;Benbadis et al, 2009;Gerber et al, 2008;Azuma et al, 2003), and given that the findings are not noted down using a set category of outcomes or that no commonly accepted guidelines exist for describing some properties, reports become di icult to query and compare to the findings of other clinicians. In recent work by Beniczky et al (2013), the authors describe a set of guidelines and definitions (including the reporting of common background properties) that is being constructed as part of a pan-European project with the goal of providing more consistency and structure for the reports in clinical EEG reviews (Beniczky et al, 2013;Aurlien et al, 2004Aurlien et al, , 2007.…”

“…Unfortunately, various studies have shown that a large inter-and intra-observer variability still exists between reviewers. Depending on the reported feature or decision outcome, the inter-rater agreement (Kappa coe icients) range from slight (0.09) to substantial (0.94) (Haut et al, 2002;Benbadis et al, 2009;Gerber et al, 2008;Azuma et al, 2003). One of the main reasons for this is a lack of consistency in describing the properties accurately.…”