The suitability of a quasi-Newton algorithm for estimating fitness-fatigue models: Sensitivity, troublesome local optima, and implications for future research (An in silico experimental design)

Abstract

Fitting an FFM via NLS in practice assumes that a unique optimal solution exists and can be found by the algorithm applied. However, this idealistic scenario may not hold for two reasons: 1) the absolute minimum may not be unique; and 2) local minima, saddle points, and/or plateau features may exist that cause problems for certain algorithms. If there exist different parameter sets in the domain that share the same global minimum under standard NLS, then there is a situation where parameters aren’t uniquely identified without additional constraints or regularisation terms. However, more likely is that problems with the typical FFM fitting process will stem from the existence of local minima, saddles, or plateau features that cause the algorithm to converge to a solution not equal to the global minimum. Local optima can provoke sensitivities in the fitting process for first and second-order algorithms that are by definition local optimisers. This manifests as sensitivity to initial parameter estimates (i.e., the starting point the algorithm initialises the search from). The extent of starting point sensitivity is largely unknown in the context of FFMs for common algorithms adopted and has not been studied directly. Given this concern, research reporting a single model solution derived from ‘one shot’ minimisation of NLS via typical first and second-order algorithms is fundamentally limited by possible uncertainty as to the suitability of fitted estimates as global minimisers. Therefore, the primary aim of this study was to investigate the sensitivity of a classical first-order search algorithm to selection of initial estimates when fitting a fitness-fatigue model (FFM) via nonlinear least-squares (NLS), and to subsequently assess the existence of local optima. A secondary aim of this study was to examine the implications of any findings in relation to previous research and provide considerations for future experimentation. The aims of the study were addressed through a computer experiment (in silico) approach that adopted a deterministic assumption the FFM completely specified athlete response. Under this assumption, two FFMs (standard, and fitness-delay) were simulated under a set of hypothetical model inputs and manually selected ‘true’ parameter values (for each FFM), generating a set of synthetic performance data. The two FFMs were refitted to the synthetic performance data without noise (and under the same model inputs) by the quasi-Newton L-BFGS-B algorithm in a repetitive fashion initiated from multiple starting points in the parameter space, attempting to at each search recover the true parameter values. Estimates obtained from this process were then further transformed into prediction errors quantifying in-sample model fit across the iterations and non-true solutions. Within the standard model scenarios, 69.1-70.3% of solutions found were the true parameters. In contrast, within the fitness-delay model scenarios, 17.6-17.9% of solutions found were the true parameters. A large number of unique non-true solutions were found for both the standard model (N=275-353) and the fitness-delay model (N=383-550) in this idealistic environment. Many of the non-true extrema found by the algorithm were local minima or saddles. Strong in-sample model fit was also observed across non-true solutions for both models. Collectively, these results indicate the typical NLS approach to fitting FFMs is harder for a hill-climbing algorithm to solve than previously recognised in the literature, particularly for models of higher complexity. The findings of this study add weight to the hypothesis that there exists substantial doubt in reported estimates across prior literature where local optimisers have been used or models more complex than the standard FFM applied, particularly when optimisation procedures reported have lacked the relevant detail to indicate that these issues have been considered. Future research should consider the use of global optimisation algorithms, hybrid approaches, or different perspectives (e.g., Bayesian optimisation).

Validity and Accuracy of Impulse-Response Models for Modeling and Predicting Training Effects on Performance of Swimmers

“…The present findings are that Model Ba and Model Bu were relevant for modeling the change in performance with training in athletes and showed a somewhat lesser ability to predict individual future performance from past data in a given swimmer. Impulse-response models suffer, however, from some limitations indicated in a series of recent studies (3,(9)(10)(11).…”

“…The coefficient of variation of 0.40% ± 0.23% was close to the value around 0.9% observed for field tests of sprint running (26). One limitation is the existence of local minima, resulting in convergence to a solution sensitive to the initial parameters chosen for the starting point of the algorithm (10,11). In our opinion, this troublesome problem is particularly relevant when all the parameters are estimated together using a typical fitting process.…”

“…On the other hand, the data must also provide results of performance tests done regularly during the intervention so that the model can be parameterized and we can test its ability to describe the relationship between training and performance. It has been shown that measurement reliability and testing frequency (9) and methods for parameter estimation (10,11) can affect model accuracy. The number of athletes providing the data and the duration of data collection are also key to producing a powerful analysis.…”