Separable Nonlinear Least-Squares Parameter Estimation for Complex Dynamic Systems

Abstract

Nonlinear dynamic models are widely used for characterizing functional forms of processes that govern complex biological pathway systems. Over the past decade, validation and further development of these models became possible due to data collected via high-throughput experiments using methods from molecular biology. While these data are very beneficial, they are typically incomplete and noisy, so that inferring parameter values for complex dynamic models is associated with serious computational challenges. Fortunately, many biological systems have embedded linear mathematical features, which may be exploited, thereby improving fits and leading to better convergence of optimization algorithms.In this paper, we explore options of inference for dynamic models using a novel method of separable nonlinear least-squares optimization, and compare its performance to the traditional nonlinear least-squares method. The numerical results from extensive simulations suggest that the proposed approach is at least as accurate as the traditional nonlinear least-squares, but usually superior, while also enjoying a substantial reduction in computational time.

Inference and Validation of the Structure of Lotka-Volterra Models

“…This GMA format is incomparably richer but loses many of the genuine advantages of the LV system. To some degree, the linearity of the process structure may be exploited for estimation purposes [61], but nothing comparable to the ALVI method is available.…”

Differential equations in data analysis

“…It seems that although the statistical problem of parameter estimation of ODEs has been widely studied, powerful mathematical devices, such as the separability of ODE systems, while not new, have only recently witnessed significant advances. Our review suggests that separability is especially useful when learning high‐dimensional ODE systems, and thus should be further explored from both a theoretical and a methodological perspective; see Dattner et al (2020) for an overview and suggestions for future research. However, unlike the statistical problem of parameter estimation, the other scientific tasks discussed in this review, such as the design of experiments, model selection, discovering governing equations, and causality analysis, seem to be less well developed.…”

“…Indeed, using a direct integral approach, Dattner et al (2017) applied a SLS technique to the inference of parameters in a predator–prey system acting in a heterogeneous environment, while Wu et al (2019) used separability to estimate the parameters of high‐dimensional linear ODE systems. Recently, Dattner, Ship, and Voit (2020) further studied the early ideas of Varah (1982) regarding separating estimation tasks into linear and nonlinear aspects, where nonlinear optimization is used only to estimate the nonlinear parameters θ NL , which, in comparison to the NLS approach, can substantially reduce the dimensionality of the nonlinear optimization problem. Indeed, the NLS solution does not take into account the specific linear form of the ODEs, but uses the general form in (1).…”

“…Thus, SLS problems are a special class of NLS problems. Dattner et al (2020) explored inference for dynamic models using SLS and NLS, and concluded that the SLS method is at least as accurate as traditional nonlinear least squares, but is usually superior, while also enjoying a substantial reduction in computational time. Recently, Ranciati, Wit, and Viroli (2019) studied separable ODEs using a Bayesian framework.…”

Kinematic Calibration of Industrial Robots Based on Distance Information Using a Hybrid Identification Method

“…Unlike coordinates, distance is a relative value, which is always the same in different coordinate systems without any transformation. e third step, identification, actually is an optimization problem, in which the least square (LS) algorithm [24][25][26][27] is the most commonly used way. But LS is easily affected by the noise of data, which can lead to failure in identification.…”