Large-Scale Automated Sleep Staging

Sun, Haoqi; Jia, Jian; Goparaju, Balaji; Huang, Guang-Bin; Sourina, Olga; Bianchi, Matt T.; Westover, M. Brandon

doi:10.1093/sleep/zsx139

Cited by 89 publications

(72 citation statements)

References 30 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To assess the impact of providing information about sleep stages, we use a recently published algorithm that performs at a level similar to human experts in assigning sleep stages to consecutive 30 second epochs of EEG (Sun et al 2017). The model outputs a probability for stages NREM1, NREM2, NREM3, REM or awake.…”

Section: Methodsmentioning

confidence: 99%

Detecting abnormal electroencephalograms using deep convolutional networks

Leeuwen

Sun

Tabaeizadeh

et al. 2019

Clinical Neurophysiology

Self Cite

View full text Add to dashboard Cite

Objectives: Electroencephalography (EEG) is a central part of the medical evaluation for patients with neurological disorders. Training an algorithm to label the EEG normal vs abnormal seems challenging, because of EEG heterogeneity and dependence of contextual factors, including age and sleep stage. Our objectives were to validate prior work on an independent data set suggesting that deep learning methods can discriminate between normal vs abnormal EEGs, to understand whether age and sleep stage information can improve discrimination, and to understand what factors lead to errors. Methods: We train a deep convolutional neural network on a heterogeneous set of 8,522 routine EEGs from the Massachusetts General Hospital. We explore several strategies for optimizing model performance, including accounting for age and sleep stage. Results: The area under the receiver operating characteristic curve (AUC) on an independent test set (n = 851) is 0.917 marginally improved by including age (AUC=0.924), and both age and sleep stages (AUC= 0.925), though not statistically significant. Conclusions: The model architecture generalizes well to an independent dataset. Adding age and sleep stage to the model does not significantly improve performance. Significance: Insights learned from misclassified examples, and minimal improvement by adding sleep stage and age suggest fruitful directions for further research.

show abstract

Section: Methodsmentioning

confidence: 99%

Detecting abnormal electroencephalograms using deep convolutional networks

Leeuwen

Sun

Tabaeizadeh

et al. 2019

Clinical Neurophysiology

Self Cite

View full text Add to dashboard Cite

show abstract

“…We also compared DeepSleep with recent state-of-the-art methods in sleep stage scoring. These methods extracted features from 30-second epochs through short-time Fourier transform (STFT) 27,28 or Thomson’s multitaper 25,29 . They were originally designed for automatic sleep staging and we applied them to the task of detecting sleep arousals on the same 2018 PhysioNet data.…”

Section: Resultsmentioning

confidence: 99%

DeepSleep: Fast and Accurate Delineation of Sleep Arousals at Millisecond Resolution by Deep Learning

Guan

2019

Preprint

View full text Add to dashboard Cite

Sleep arousals are transient periods of wakefulness punctuated into sleep. Excessive sleep arousals are associated with many negative effects including daytime sleepiness and sleep disorders. High-quality annotation of polysomnographic recordings is crucial for the diagnosis of sleep arousal disorders. Currently, sleep arousals are mainly annotated by human experts through looking at millions of data points manually, which requires considerable time and effort. Here we present a deep learning approach, DeepSleep, which ranked first in the 2018 PhysioNet Challenge for automatically segmenting sleep arousal regions based on polysomnographic recordings. DeepSleep features accurate (area under receiver operating characteristic curve of 0.93), high-resolution (5-millisecond resolution), and fast (10 seconds per sleep record) delineation of sleep arousals.

show abstract

“…48 This should not necessarily affect expectations in terms of reliability. Published methods also differ by the way they are assessed, 39, 42–46, 49, 50 study protocol (population studied and number of experts) and comparison methodology (reference setup and statistics). Providing public sleep databases is an ongoing and useful process for several years in the USA (www.sleepdata.org 35 ), in Europe (www.physionet.org 51 ) and more recently in Canada (www.ceams-carsm.ca/en/MASS 52 ).…”

Section: Discussionmentioning

confidence: 99%

Looking for a reference for large datasets: relative reliability of visual and automatic sleep scoring

Muto

Berthomier

Schmidt

et al. 2019

Preprint

View full text Add to dashboard Cite

Study ObjectivesNew challenges in sleep science require to describe fine grain phenomena or to deal with large datasets. Beside the human resource challenge of scoring huge datasets, the inter- and intra-expert variability may also reduce the sensitivity of such studies. Searching for a way to disentangle the variability induced by the scoring method from the actual variability in the data, visual and automatic sleep scorings of healthy individuals were examined.MethodsA first dataset (DS1, 4 recordings) scored by 6 experts plus an autoscoring algorithm was used to characterize inter-scoring variability. A second dataset (DS2, 88 recordings) scored a few weeks later was used to investigate intra-expert variability. Percentage agreements and Conger’s kappa were derived from epoch-by-epoch comparisons on pairwise, consensus and majority scorings.ResultsOn DS1 the number of epochs of agreement decreased when the number of expert increased, in both majority and consensus scoring, where agreement ranged from 86% (pairwise) to 69% (all experts). Adding autoscoring to visual scorings changed the kappa value from 0.81 to 0.79. Agreement between expert consensus and autoscoring was 93%. On DS2 intra-expert variability was evidenced by the kappa systematic decrease between autoscoring and each single expert between datasets (0.75 to 0.70).ConclusionsVisual scoring induces inter- and intra-expert variability, which is difficult to address especially in big data studies. When proven to be reliable and if perfectly reproducible, autoscoring methods can cope with intra-scorer variability making them a sensible option when dealing with large datasets.Statement of SignificanceWe confirmed and extended previous findings highlighting the intra- and inter-expert variability in visual sleep scoring. On large datasets those variability issues cannot be completely addressed by neither practical nor statistical solutions such as group training, majority or consensus scoring.When an automated scoring method can be proven to be as reasonably imperfect as visual scoring but perfectly reproducible, it can serve as a reliable scoring reference for sleep studies.

show abstract

Large-Scale Automated Sleep Staging

Abstract: Training with a large data set enables automated sleep staging that compares favorably with human scorers. Because testing was performed on a large and heterogeneous data set, the performance estimate has low variance and is likely to generalize broadly.

Cited by 89 publications

References 30 publications

Detecting abnormal electroencephalograms using deep convolutional networks

Detecting abnormal electroencephalograms using deep convolutional networks

DeepSleep: Fast and Accurate Delineation of Sleep Arousals at Millisecond Resolution by Deep Learning

Looking for a reference for large datasets: relative reliability of visual and automatic sleep scoring

Contact Info

Product

Resources

About