From Pulses to Sleep Stages: Towards Optimized Sleep Classification Using Heart-Rate Variability

Abstract

More and more people quantify their sleep using wearables and are becoming obsessed in their pursuit of optimal sleep (“orthosomnia”). However, it is criticized that many of these wearables are giving inaccurate feedback and can even lead to negative daytime consequences. Acknowledging these facts, we here optimize our previously suggested sleep classification procedure in a new sample of 136 self-reported poor sleepers to minimize erroneous classification during ambulatory sleep sensing. Firstly, we introduce an advanced interbeat-interval (IBI) quality control using a random forest method to account for wearable recordings in naturalistic and more noisy settings. We further aim to improve sleep classification by opting for a loss function model instead of the overall epoch-by-epoch accuracy to avoid model biases towards the majority class (i.e., “light sleep”). Using these implementations, we compare the classification performance between the optimized (loss function model) and the accuracy model. We use signals derived from PSG, one-channel ECG, and two consumer wearables: the ECG breast belt Polar® H10 (H10) and the Polar® Verity Sense (VS), an optical Photoplethysmography (PPG) heart-rate sensor. The results reveal a high overall accuracy for the loss function in ECG (86.3 %, κ = 0.79), as well as the H10 (84.4%, κ = 0.76), and VS (84.2%, κ = 0.75) sensors, with improvements in deep sleep and wake. In addition, the new optimized model displays moderate to high correlations and agreement with PSG on primary sleep parameters, while measures of reliability, expressed in intra-class correlations, suggest excellent reliability for most sleep parameters. Finally, it is demonstrated that the new model is still classifying sleep accurately in 4-classes in users taking heart-affecting and/or psychoactive medication, which can be considered a prerequisite in older individuals with or without common disorders. Further improving and validating automatic sleep stage classification algorithms based on signals from affordable wearables may resolve existing scepticism and open the door for such approaches in clinical practice.

Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep

“…The Cohen's κ coefficient of 0.76 indicated substantial agreement between sleep 2 with VS and PSG. A similar pattern of results was observed in the group-level epoch-by-epoch analysis for the H10 (also see Topalidis et al, 2023b), although the number of analysed recordings was considerably lower (cf. Figure 2).…”

“…Wake detection was the most challenging stage (74%), consistent with the known difficulty of separating quiet wake from light sleep across cardiac-based systems. The model's precision and recall profiles -both high and well balanced -reflect the successful correction of stage-specific imbalances reported in the optimised model (Topalidis et al, 2023b). Overall, sleep 2 -VS displayed substantial agreement with PSG supported by detailed multi-stage sleep classification.…”

Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep

“…The Cohen's κ coefficient of 0.76 indicated substantial agreement between sleep 2 with VS and PSG. A similar pattern of results was observed in the group-level epoch-by-epoch analysis for the H10 (also see Topalidis et al, 2023b), although the number of analysed recordings was considerably lower (cf. Figure 2).…”

“…Wake detection was the most challenging stage (74%), consistent with the known difficulty of separating quiet wake from light sleep across cardiac-based systems. The model's precision and recall profiles -both high and well balanced -reflect the successful correction of stage-specific imbalances reported in the optimised model (Topalidis et al, 2023b). Overall, sleep 2 -VS displayed substantial agreement with PSG supported by detailed multi-stage sleep classification.…”

Mamba-based deep learning approach for sleep staging on a wireless multimodal wearable system without electroencephalography

“…This is not surprising since signal quality is often much higher for attended in-laboratory PSGs where more intrusive sensors can be used, signal quality is continuously monitored, and issues can be addressed by the technologist as they occur. Even so, despite using wearable rather than PSG sensor data, the macro performance of our ensemble RNN model approached [ 21–23 , 25 , 26 ] or exceeded [ 24 , 27 ] the performance reported for models trained on much larger sets of PSG data.…”

“…When we limit our analyses to younger individuals without sleep apnea or significant periodic limb movements, model performance improves further ( Figure 5 ). Another contrast between [ 25 ] and the present study is that [ 25 ] performed much stricter QC for recording quality, with 54/136 or nearly 40 per cent of nights of wearable sensor recording excluded from analysis due to poor signal quality. In contrast, in the interest of evaluating real-world performance, we did not exclude any recording or any epochs on the basis of signal quality, and the reported results reflect consideration of all available epochs of recording, irrespective of signal quality.…”

“…Our model exceeded [ 20 , 22 , 24 , 28 , 29 ] or matched [ 41 , 43 ] the benchmarks of most wearable sensor-based models with the exception of one [ 25 ]. Of note, this study examined a younger set of participants (mean age 45) who were free of psychiatric and neurological co-morbidity, which may influence the relationship between sleep stage and its autonomic, cardiac, and pulmonary manifestations.…”

Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep

“…The Cohen's κ coefficient of 0.76 indicated substantial agreement between sleep 2 with VS and PSG. A similar pattern of results was observed in the group-level epoch-by-epoch analysis for the H10 (also see Topalidis et al, 2023b), although the number of analysed recordings was considerably lower (cf. Figure 2).…”

“…Wake detection was the most challenging stage (74%), consistent with the known difficulty of separating quiet wake from light sleep across cardiac-based systems. The model's precision and recall profiles -both high and well balanced -reflect the successful correction of stage-specific imbalances reported in the optimised model (Topalidis et al, 2023b). Overall, sleep 2 -VS displayed substantial agreement with PSG supported by detailed multi-stage sleep classification.…”

Mamba-based deep learning approach for sleep staging on a wireless multimodal wearable system without electroencephalography

“…This is not surprising since signal quality is often much higher for attended in-laboratory PSGs where more intrusive sensors can be used, signal quality is continuously monitored, and issues can be addressed by the technologist as they occur. Even so, despite using wearable rather than PSG sensor data, the macro performance of our ensemble RNN model approached [ 21–23 , 25 , 26 ] or exceeded [ 24 , 27 ] the performance reported for models trained on much larger sets of PSG data.…”

“…When we limit our analyses to younger individuals without sleep apnea or significant periodic limb movements, model performance improves further ( Figure 5 ). Another contrast between [ 25 ] and the present study is that [ 25 ] performed much stricter QC for recording quality, with 54/136 or nearly 40 per cent of nights of wearable sensor recording excluded from analysis due to poor signal quality. In contrast, in the interest of evaluating real-world performance, we did not exclude any recording or any epochs on the basis of signal quality, and the reported results reflect consideration of all available epochs of recording, irrespective of signal quality.…”

“…Our model exceeded [ 20 , 22 , 24 , 28 , 29 ] or matched [ 41 , 43 ] the benchmarks of most wearable sensor-based models with the exception of one [ 25 ]. Of note, this study examined a younger set of participants (mean age 45) who were free of psychiatric and neurological co-morbidity, which may influence the relationship between sleep stage and its autonomic, cardiac, and pulmonary manifestations.…”

Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep

“…The Cohen's κ coefficient of 0.76 indicated substantial agreement between sleep 2 with VS and PSG. A similar pattern of results was observed in the group-level epoch-by-epoch analysis for the H10 (also see Topalidis et al, 2023b), although the number of analysed recordings was considerably lower (cf. Figure 2).…”

“…Wake detection was the most challenging stage (74%), consistent with the known difficulty of separating quiet wake from light sleep across cardiac-based systems. The model's precision and recall profiles -both high and well balanced -reflect the successful correction of stage-specific imbalances reported in the optimised model (Topalidis et al, 2023b). Overall, sleep 2 -VS displayed substantial agreement with PSG supported by detailed multi-stage sleep classification.…”

Mamba-based deep learning approach for sleep staging on a wireless multimodal wearable system without electroencephalography

“…This is not surprising since signal quality is often much higher for attended in-laboratory PSGs where more intrusive sensors can be used, signal quality is continuously monitored, and issues can be addressed by the technologist as they occur. Even so, despite using wearable rather than PSG sensor data, the macro performance of our ensemble RNN model approached [ 21–23 , 25 , 26 ] or exceeded [ 24 , 27 ] the performance reported for models trained on much larger sets of PSG data.…”

“…When we limit our analyses to younger individuals without sleep apnea or significant periodic limb movements, model performance improves further ( Figure 5 ). Another contrast between [ 25 ] and the present study is that [ 25 ] performed much stricter QC for recording quality, with 54/136 or nearly 40 per cent of nights of wearable sensor recording excluded from analysis due to poor signal quality. In contrast, in the interest of evaluating real-world performance, we did not exclude any recording or any epochs on the basis of signal quality, and the reported results reflect consideration of all available epochs of recording, irrespective of signal quality.…”

“…Our model exceeded [ 20 , 22 , 24 , 28 , 29 ] or matched [ 41 , 43 ] the benchmarks of most wearable sensor-based models with the exception of one [ 25 ]. Of note, this study examined a younger set of participants (mean age 45) who were free of psychiatric and neurological co-morbidity, which may influence the relationship between sleep stage and its autonomic, cardiac, and pulmonary manifestations.…”