The probability of false positives in zero-dimensional analyses of one-dimensional kinematic, force and EMG trajectories

Abstract

A false positive is the mistake of inferring an e↵ect when none exists, and although ↵ controls the false positive (Type I error) rate in classical hypothesis testing, a given ↵ value is accurate only if the underlying model of randomness appropriately reflects experimentally observed variance. Hypotheses pertaining to one-dimensional (1D) (e.g. time-varying) biomechanical trajectories are most often tested using a traditional zero-dimensional (0D) Gaussian model of randomness, but variance in these datasets variance is clearly 1D. The purpose of this study was to determine the likelihood that analyzing smooth 1D data with a 0D model of variance will produce false positives. We first used random field theory (RFT) to predict the probability of false positives in 0D analyses. We then validated RFT predictions via numerical simulations of smooth Gaussian 1D trajectories. Results showed that, across a range of public kinematic, force and EMG datasets, the median false positive rate was 0.382 and not the assumed ↵=0.05, even for a simple two-sample t test involving N =10 trajectories per group. The median false positive rates for experiments involving three-component vector trajectories was p=0.764. This rate increased to p=0.945 for two three-component vector trajectories, and to p=0.999 for six three-component vectors. This implies that experiments involving vector trajectories have a high probability of yielding 0D statistical significance when there is, in fact, no 1D e↵ect. Either (a) explicit a priori identification of 0D metrics or (b) adoption of 1D methods can more tightly control ↵.

The effect of stride length on lower extremity joint kinetics at various gait speeds

“…peak torque, range of motion, etc. ), definition of confidence intervals and control of false positive rates (FPR) requires proper correction for multiple comparisons that accounts for the temporal correlations in the input time series [23]. We used the software SPM1D, a parametric statistical testing method developed for nD time series [24], to control for FPR in the analysis of normalized joint moment profiles , and quantify the effect of both factors (GS and SL) and of their interaction on the dependent variable in different phases of the gait cycle.…”

Region-of-interest analyses of one-dimensional biomechanical trajectories: bridging 0D and 1D theory, augmenting statistical power

“…An investigator must therefore balance local signal detectability (via narrow a priori ROI definition) with full-field signal detectability (via larger ROIs). In the absence of an a priori hypothesis regarding a specific portion of the continuum it has been suggested that full-field analyses should be conducted (Pataky, Vanrenterghem & Robinson, 2016), and this paper extends that suggestion to include a caveat for exploratory work in a two-stage procedure involving an initial full-field analysis on an exploratory dataset followed by more precise ROI-based testing (Fig. 6).…”

“…We would therefore recommend that, instead of choosing a single ROI size or a single α , investigators should actively manipulate all parameters that might affect ultimate conclusions including: ROI size, α , data filtering, coordinate system definitions, etc., and then actively report the results of those manipulations in a sensitivity analysis as has been done elsewhere (Pataky et al, 2014). If the reported results are robust to those manipulations then one can be more confident that the reported results are neither false positives (Pataky, Vanrenterghem & Robinson, 2016) nor false negatives. Such manipulations may be especially important for biomechanical datasets considering that these data can be sensitive to ROI definitions (Figs.…”

“…ROI size was systematically varied over the full range of possibilities (i.e., from 0 to 100% trajectory length). FWHM and ν values were selected to span a range of values observed in representative open-access biomechanical datasets (FWHM = [5, 50], ν  = [5, 49]) (Pataky, Vanrenterghem & Robinson, 2016). …”

“…2B and 2C) (Pataky, Robinson & Vanrenterghem, 2013) ( Q  = 101) contained stance-phase body-weight-normalized vertical ground reaction forces (VGRF) from seven subjects who each performed normal, self-paced walking and fast walking in a randomized order. Analysis proceeded in two-stages to demonstrate how ROIs can be used to increase analysis sensitivity in exploratory experiments (Pataky, Vanrenterghem & Robinson, 2016, Fig.7). First, the chronologically first subject was separated as a pilot subject (Fig.…”