Validation of Fitbit Charge 2 Sleep and Heart Rate Estimates Against Polysomnographic Measures in Shift Workers: Naturalistic Study

Abstract

Background Multisensor fitness trackers offer the ability to longitudinally estimate sleep quality in a home environment with the potential to outperform traditional actigraphy. To benefit from these new tools for objectively assessing sleep for clinical and research purposes, multisensor wearable devices require careful validation against the gold standard of sleep polysomnography (PSG). Naturalistic studies favor validation. Objective This study aims to validate the Fitbit Charge 2 against portable home PSG in a shift-work population composed of 59 first responder police officers and paramedics undergoing shift work. Methods A reliable comparison between the two measurements was ensured through the data-driven alignment of a PSG and Fitbit time series that was recorded at night. Epoch-by-epoch analyses and Bland-Altman plots were used to assess sensitivity, specificity, accuracy, the Matthews correlation coefficient, bias, and limits of agreement. Results Sleep onset and offset, total sleep time, and the durations of rapid eye movement (REM) sleep and non–rapid-eye movement sleep stages N1+N2 and N3 displayed unbiased estimates with nonnegligible limits of agreement. In contrast, the proprietary Fitbit algorithm overestimated REM sleep latency by 29.4 minutes and wakefulness after sleep onset (WASO) by 37.1 minutes. Epoch-by-epoch analyses indicated better specificity than sensitivity, with higher accuracies for WASO (0.82) and REM sleep (0.86) than those for N1+N2 (0.55) and N3 (0.78) sleep. Fitbit heart rate (HR) displayed a small underestimation of 0.9 beats per minute (bpm) and a limited capability to capture sudden HR changes because of the lower time resolution compared to that of PSG. The underestimation was smaller in N2, N3, and REM sleep (0.6-0.7 bpm) than in N1 sleep (1.2 bpm) and wakefulness (1.9 bpm), indicating a state-specific bias. Finally, Fitbit suggested a distribution of all sleep episode durations that was different from that derived from PSG and showed nonbiological discontinuities, indicating the potential limitations of the staging algorithm. Conclusions We conclude that by following careful data processing processes, the Fitbit Charge 2 can provide reasonably accurate mean values of sleep and HR estimates in shift workers under naturalistic conditions. Nevertheless, the generally wide limits of agreement hamper the precision of quantifying individual sleep episodes. The value of this consumer-grade multisensor wearable in terms of tackling clinical and research questions could be enhanced with open-source algorithms, raw data access, and the ability to blind participants to their own sleep data.

Performance of a Multisensor Smart Ring to Evaluate Sleep: In-Lab and Home-Based Evaluation Relative to Polysomnography and Actigraphy: Importance of Generalized Versus Personalized Scoring

“…This commercially available device uses both accelerometry and PPG heart rate data to estimate sleep and wake 30 . Previous studies have shown that the Fitbit device is a valid and reliable tool for assessment of sleep, with strong agreement compared with PSG (sleep-wake sensitivity ≥0.93) 28,30,40,41 . Third, participants were asked to wear a Whoop device (Whoop 3.0, Whoop Inc.).…”

Sleep and seizure risk in epilepsy: Bed and wake times are more important than sleep duration

“…Recent studies have shown that newer Fitbit devices, which rely on multiple signals to detect sleep stages, have satisfactory performance with regards to their total sleep duration estimations and the transition from wake to sleep and sleep to wake. 32,33 However, for classification of sleep stages, wearable smartwatches are not yet suitable substitutes for polysomnography, 22,23,34 with a few exceptions in their ability to detect transitioning from certain states (deep sleep to wake state, and light sleep to REM sleep) and the likelihood of remaining in REM sleep. 34 Despite this lack of confidence in classification of sleep stages, it is still possible to detect clear individual-specific relationships between seizures and other sleep parameters, notably total sleep duration, sleep onset and offset, and oversleep and undersleep, which indirectly relate to sleep architecture and sleep quality.…”

“…The copyright holder for this preprint this version posted August 6, 2022 This study did not focus on sleep composition due to limitations in the wearable sleep staging algorithm for some people 22,23,34 , although some subjects did appear to have differences in their sleep composition prior to seizure days compared to seizure-free days (Supplementary Figure 4 and Supplementary Table 5). Differences in sleep composition prior to seizures were highly individual-specific and likely reflected changes in sleep duration in many cases.…”

“…Sleep data (sleep stages and durations) was provided by Fitbit using a proprietary sleep algorithm, which estimates sleep stages (categorised into Light, Deep and REM sleep) using a combination of movement (via accelerometery) and heart rate (via photoplethysmography) patterns, including heart rate variability. According to recent studies, Fitbit adequately estimates total sleep duration, sleep onset and offset, and durations of sleep stages in most cases 22,23 ; however it is not yet a suitable substitute for gold-standard polysomnography, thus sleep stage analyses have primarily been included as supplemental data in this work. Using Fitbit, light sleep approximately corresponds to N1/N2 stages, deep sleep approximately corresponds to N3 stages, and REM sleep marked by Fitbit reflects the more autonomically unstable stage of sleep (still referred to as ‘REM’ in this study for clarity).…”

“…Sleep data (sleep stages and durations) was provided by Fitbit using a proprietary sleep algorithm, which estimates sleep stages (categorised into Light, Deep and REM sleep) using a combination of movement (via accelerometery) and heart rate (via photoplethysmography) patterns, including heart rate variability. According to recent studies, Fitbit adequately estimates total sleep duration, sleep onset and offset, and durations of sleep stages in most cases 22,23…”