An Opinion Regarding Equivalence Testing for Evaluating Measurement Agreement

Abstract

The novel statistical approach 'equivalence testing' has been proposed in order to statistically examine agreement between different physical activity measures. By using this method, researchers argued that it is possible to determine whether a method is significantly equivalent to another method. Recently, equivalence testing was supported with the use of 90% confidence interval, obtained from a mixed ANOVA, which I believe is a more robust approach. This paper further discusses the use of this method in comparison to a more well-established statistical analysis (i.e. mixed design ANOVA), as well as various limitations and arbitrary assumptions in order to perform this analysis. The paper concludes with some remarks and considerations for future use in similar approaches.

Validity of Apple Watch, Garmin Forerunner® 935 and GENEActiv for estimating energy expenditure during close quarter battle training in Special Forces soldiers

“…Further, both fixed and proportional bias, as well as unacceptably high MAPE were found, suggesting a lack of agreement between the two devices on an individual level, similar to previous findings relating to the accuracy of EE estimates from wearable devices (Shei et al., 2022). This divergence in the assessment of agreement is not unexpected as the statistical methods used assess different components of agreement (group‐level vs. individual‐level) (Adamakis, 2019), but does highlight the need to interpret the results of studies investigating the validity of wearable devices carefully. Of note in this study, the MAPE of 11.3% for the Apple Watch, which was the lowest of the devices tested in this study, was similar to the error found in other studies using protocols of fast walking or running, and substantially lower than the MAPE reported in studies estimating EE at rest.…”