Accuracy and Precision of Consumer-Grade Wearable Activity Monitors for Assessing Time Spent in Sedentary Behavior in Children and Adolescents: Systematic Review

Abstract

Background A large number of wearable activity monitor models are released and used each year by consumers and researchers. As more studies are being carried out on children and adolescents in terms of sedentary behavior (SB) assessment, knowledge about accurate and precise monitoring devices becomes increasingly important. Objective The main aim of this systematic review was to investigate and communicate findings on the accuracy and precision of consumer-grade physical activity monitors in assessing the time spent in SB in children and adolescents. Methods Searches of PubMed (MEDLINE), Scopus, SPORTDiscus (full text), ProQuest, Open Access Theses and Dissertations, DART Europe E-theses Portal, and Networked Digital Library of Theses and Dissertations electronic databases were performed. All relevant studies that compared different types of consumer-grade monitors using a comparison method in the assessment of SB, published in European languages from 2015 onward were considered for inclusion. The risk of bias was estimated using Consensus-Based Standards for the Selection of Health Status Measurement Instruments. For enabling comparisons of accuracy measures within the studied outcome domain, measurement accuracy interpretation was based on group mean or percentage error values and 90% CI. Acceptable limits were predefined as –10% to +10% error in controlled and free-living settings. For determining the number of studies with group error percentages that fall within or outside one of the sides from previously defined acceptable limits, two 1-sided tests of equivalence were carried out, and the direction of measurement error was examined. Results A total of 8 studies complied with the predefined inclusion criteria, and 3 studies provided acceptable data for quantitative analyses. In terms of the presented accuracy comparisons, 14 were subsequently identified, with 6 of these comparisons being acceptable in terms of quantitative analysis. The results of the Cochran Q test indicated that the included studies did not share a common effect size (Q5=82.86; P<.001). I2, which represents the percentage of total variation across studies due to heterogeneity, amounted to 94%. The summary effect size based on the random effects model was not statistically significant (effect size=14.36, SE 12.04, 90% CI −5.45 to 34.17; P=.23). According to the equivalence test results, consumer-grade physical activity monitors did not generate equivalent estimates of SB in relation to the comparison methods. Majority of the studies (3/7, 43%) that reported the mean absolute percentage errors have reported values of <30%. Conclusions This is the first study that has attempted to synthesize available evidence on the accuracy and precision of consumer-grade physical activity monitors in measuring SB in children and adolescents. We found very few studies on the accuracy and almost no evidence on the precision of wearable activity monitors. The presented results highlight the large heterogeneity in this area of research. Trial Registration PROSPERO CRD42021251922; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=251922

Exploring needs and requirements for a prototype device measuring physical activity in pediatric physical therapy: A qualitative study

“…Compared to self-report instruments, device-based instruments like wearable activity monitors (AMs) appear to have stronger psychometric properties [10]. The market of consumer grade AMs is growing rapidly but literature is inconclusive about the psychometric properties of consumer grade AMs to measure free-living physical behavior (e.g., PA categories or sedentary behavior) consistently in children and adolescents [15][16][17]. Furthermore, PPTs need to be able to analyze the data which is complicated with consumer-grade wearables such as the Fitbit [18].…”

A Device Agnostic Approach to Predict Children’s Activity from Consumer Wearable Accelerometer Data: A Proof-of-Concept Study

“…This approach has the potential to leverage the strengths of consumer wearable devices while overcoming the major limitation associated with these devices. All previous studies have examined the validity of the physical activity metrics produced by the proprietary algorithms of consumer wearables (10)(11)(12), whereas no studies have explored the validity of applying a device-agnostic approach of predicting activity intensity levels of children using consumer wearable devices. Therefore, the purpose of this proof-of-concept study was to explore the potential of a device agnostic approach when applied to consumer wearable devices and to evaluate the agreement of physical activity estimates produced by raw accelerometry data collected via two widely used consumer wearable devices and a research-grade device compared with a criterion of indirect calorimetry.…”