Validity, potential clinical utility, and comparison of consumer and research‐grade activity trackers in Insomnia Disorder I: In‐lab validation against polysomnography

Abstract

SummaryConsumer activity trackers claiming to measure sleep/wake patterns are ubiquitous within clinical and consumer settings. However, validation of these devices in sleep disorder populations are lacking. We examined 1 night of sleep in 42 individuals with insomnia (mean = 49.14 ± 17.54 years) using polysomnography, a wrist actigraph (Actiwatch Spectrum Pro: AWS) and a consumer activity tracker (Fitbit Alta HR: FBA). Epoch‐by‐epoch analysis and Bland−Altman methods evaluated each device against polysomnography for sleep/wake detection, total sleep time, sleep efficiency, wake after sleep onset and sleep latency. FBA sleep stage classification of light sleep (N1 + N2), deep sleep (N3) and rapid eye movement was also compared with polysomnography. Compared with polysomnography, both activity trackers displayed high accuracy (81.12% versus 82.80%, AWS and FBA respectively; ns) and sensitivity (sleep detection; 96.66% versus 96.04%, respectively; ns) but low specificity (wake detection; 39.09% versus 44.76%, respectively; p = .037). Both trackers overestimated total sleep time and sleep efficiency, and underestimated sleep latency and wake after sleep onset. FBA demonstrated sleep stage sensitivity and specificity, respectively, of 79.39% and 58.77% (light), 49.04% and 95.54% (deep), 65.97% and 91.53% (rapid eye movement). Both devices were more accurate in detecting sleep than wake, with equivalent sensitivity, but statistically different specificity. FBA provided equivalent estimates as AWS for all traditional actigraphy sleep parameters. FBA also showed high specificity when identifying N3, and rapid eye movement, though sensitivity was modest. Thus, it underestimates these sleep stages and overestimates light sleep, demonstrating more shallow sleep than actually obtained. Whether FBA could serve as a low‐cost substitute for actigraphy in insomnia requires further investigation.

Assessing the performance of a commercial multisensory sleep tracker

“…The intrinsic sleep/wake detection sensitivity of GV4 causes a high sensitivity at the cost of a low specificity. This agrees with previous studies [9,16,32]. Classification of sleep/wake in GV4 is based on movement and HRV.…”

Performance of seven consumer sleep-tracking devices compared with polysomnography

“…Importantly, though, most of the consumer devices (five of the seven) performed either as well as or better than actigraphy on specificity, the primary indicator of a device’s wake-detection capability. The overall greatest specificity was for the Fitbit Alta HR, and it is notable that recent studies with the same device also found equal or greater specificity than actigraphy in comparison with PSG [ 27 , 29 ]. Because actigraphy is the standard technique for mobile measurement of sleep/wake and has been validated against PSG, it is reasonable to suggest that one of the best initial benchmarks for judging the validity of consumer devices should be evaluating their performance relative to actigraphy versus PSG.…”

“…Device summary measures were statistically compared with PSG using Student’s paired t -tests , Hedges ’ g effect sizes, and R 2 proportional biases. Proportional bias was calculated using linear regression methods for Bland–Altman plots [ 39 ] (and has been used previously for device performance testing [ 29 ]), which indicates the reliability of the bias versus PSG ( y -axis value) and whether it changes in proportion to the mean of the device and PSG ( x -axis value) in the Bland–Altman plots. p -values at the p < 0.05 level were considered statistically significant.…”

Sleep disturbance in post-traumatic stress disorder (PTSD): a systematic review and meta-analysis of actigraphy studies

“…Compared with the robust findings of self-reported sleep disturbance in PTSD, the documented evidence of sleep disturbance using physical assessment methods is equivocal (Babson & Feldner, 2010 ; Harvey, Jones, & Schmidt, 2003 ; Khawaja, Hashmi, Aftab, Westermeyer, & Hurwitz, 2014 ; Kobayashi, Boarts, & Delahanty, 2007 ). Polysomnography (PSG) is the gold-standard physical measure of sleep (Jumabhoy et al, 2020 ; Kushida et al, 2005 ). Incorporating multiple physiological parameters, PSG assesses the critical components of sleep architecture, including sleep stages and rapid eye movement (REM) sleep, the frequency of eye movements during REM (REM density), and the ratio of time asleep to the time spent in bed (sleep efficiency) (Boulos et al, 2019 ; Kushida et al, 2005 ).…”