Robust, automated sleep scoring by a compact neural network with distributional shift correction

Abstract

Studying the biology of sleep requires the accurate assessment of the state of experimental subjects, and manual analysis of relevant data is a major bottleneck. Recently, deep learning applied to electroencephalogram and electromyogram data has shown great promise as a sleep scoring method, approaching the limits of inter-rater reliability. As with any machine learning algorithm, the inputs to a sleep scoring classifier are typically standardized in order to remove distributional shift caused by variability in the signal collection process. However, in scientific data, experimental manipulations introduce variability that should not be removed. For example, in sleep scoring, the fraction of time spent in each arousal state can vary between control and experimental subjects. We introduce a standardization method, mixture z-scoring, that preserves this crucial form of distributional shift. Using both a simulated experiment and mouse in vivo data, we demonstrate that a common standardization method used by state-of-the-art sleep scoring algorithms introduces systematic bias, but that mixture z-scoring does not. We present a free, open-source user interface that uses a compact neural network and mixture z-scoring to allow for rapid sleep scoring with accuracy that compares well to contemporary methods. This work provides a set of computational tools for the robust automation of sleep scoring.

Automated scoring of pre-REM sleep in mice with deep learning

“…These classification performances are comparable with state-of-the-art results obtained in other studies 32 , 33 , 36 and demonstrate that the proposed network architecture is suitable for sleep scoring purposes.…”

“…However, pre-REM phases predicted by the network tended to last longer than those by the expert. When the network was trained to distinguish between the three main stages only, classification performance as quantified by the average F1 score on out-of-sample data ( , see Table 4 ) was comparable to state-of-the-art classifiers 32 , 33 . L2 and Dropout regularization as well as data augmentation increased F1 scores of the minority classes (pre-REM, artifact), while class rebalancing did not.…”

“…While research into human sleep scoring has adopted ideas and methods from deep learning 6 , 7 , there are, to our knowledge, only few studies proposing deep-learning-based systems to improve animal sleep scoring, particularly in mice. Deep neural networks introduced in two studies 32 , 33 were inspired by image recognition systems and used preprocessed spectrograms of EEG and the electromyogram (EMG) as input “images”. These systems yielded state-of-the-art classification performances but needed an additional hidden Markov model 32 that constrained the output of the system on physiologically plausible sleep state transitions or used mixture z-scoring 33 which needs an initial sample of human-annotated data for each mouse.…”

“…Deep neural networks introduced in two studies 32 , 33 were inspired by image recognition systems and used preprocessed spectrograms of EEG and the electromyogram (EMG) as input “images”. These systems yielded state-of-the-art classification performances but needed an additional hidden Markov model 32 that constrained the output of the system on physiologically plausible sleep state transitions or used mixture z-scoring 33 which needs an initial sample of human-annotated data for each mouse. Four studies proposed deep neural networks that were trained end-to-end on raw input EEG and EMG signals.…”

Sleep–Wake Cycle and EEG-Based Biomarkers during Neonate to Adult Transition in C57BL/6 Mice

“…Sleep-wake analysis. Behavioral states (wakefulness, NREM, and REM sleep) were determined by the analysis of EEG and EMG recordings with AccuSleep [16], a mouse-specific, semi-automated sleep-wake scoring algorithm written in MATLAB (The MathWorks, Natick, MA, R2018b). The epoch length was set to 2.5 s. First, 24 h signal integrity and cleanliness (lack of noise) was confirmed.…”