Manifold learning for parameter reduction

Abstract

Large scale dynamical systems (e.g. many nonlinear coupled differential equations) can often be summarized in terms of only a few state variables (a few equations), a trait that reduces complexity and facilitates exploration of behavioral aspects of otherwise intractable models. High model dimensionality and complexity makes symbolic, pen-and-paper model reduction tedious and impractical, a difficulty addressed by recently developed frameworks that computerize reduction. Symbolic work has the benefit, however, of identifying both reduced state variables and parameter combinations that matter most (effective parameters, "inputs"); whereas current computational reduction schemes leave the parameter reduction aspect mostly unaddressed. As the interest in mapping out and optimizing complex input-output relations keeps growing, it becomes clear that combating the curse of dimensionality also requires efficient schemes for input space exploration and reduction. Here, we explore systematic, data-driven parameter reduction by means of effective parameter identification, starting from current nonlinear manifoldlearning techniques enabling state space reduction. Our approach aspires to extend the data-driven determination of effective state variables with the data-driven discovery of effective model parameters, and thus to accelerate the exploration of high-dimensional parameter spaces associated with com- * These two authors contributed equally to this work.Given access to input-output information (black-box function evaluation) but no formulas, one might not even suspect that only the single parameter combination p eff = p 1 p 2 matters. Fitting the model to data f * = (1, 0, 1) in the absence of such information, one would find an entire curve in parameter space that fits the observations. A data fitting algorithm based only on function evaluations could be "confused" by such behavior in declaring convergence. As seen in Fig. 1(a), different initial conditions fed to an optimizer with a practical fitting tolerance δ ≈ 10 −3 (see figure caption for details) converge to many, widely different results tracing a level curve of p eff . The subset of good fits is effectively 1−D; more importantly, and moving beyond the fit to this particular data, the entire parameter space is foliated by such 1−D curves (neutral sets), each composed of points indistinguishable from the model output perspective. Parameter non-identifiability is therefore a structural feature of the model, not an artifact of optimization. The appropriate, intrinsic way to describe parameter space for this problem is through the effective parameter p eff and its level sets. Consider now the inset of Fig. 1(a), corresponding to the perturbed model f ε (p 1 , p 2 ) = f 0 (p 1 , p 2 ) + 2ε(p 1 − p 2 , 0, 0) and fit to the same data. Here, the parameters are identifiable and the minimizer (p 1 , p 2 ) unique: a perfect fit exists. However, the foliation observed for ε = 0 is loosely remembered in the shape of the residual level curves, and the optimizer would be co...

A quantitative framework for exploring exit strategies from the COVID-19 lockdown

“…This uncertainty is even more significant with respect to the current situation regarding the asymptomatics which, while concretely studied in some cases [26] , presents tremendous variability [43] between studies. As indicated above, the issue at hand is crucial from the point of view of modeling, both regarding issues of dimension reduction [46] , as well as those of identifiability of the models [44] .…”

“…Actually, we have shown that it is impossible to determine uniquely all 9 model parameters from a reliable set of given data. This is indeed an example of dimension reduction: this represents the crucial aspect of whether a given model outcome depends not on individual parameters alone (except for one of the model parameters) but rather on expressions formed by suitable (irreducible) combinations of parameters; for a recent, data-driven example, see [46] . In addition to the existence of the above prohibitive result, many of the needed data are unavailable.…”

Explainable Machine Learning for Scientific Insights and Discoveries

“…A broad framework leverages unsupervised learning approaches to learn low-complexity representations of physical process observations. In many cases where the underlying process features a small number of degrees of freedom, it is shown that nonlinear manifold learning algorithms are able to discern these degrees of freedoms as the component dimensions of low-dimensional nonlinear manifold embeddings, which preserve the underlying geometry of the original data space [Yair et al, 2017, Dsilva et al, 2018, Holiday et al, 2019. It can be seen that the embed-ding coordinates show relation to known physical quantities.…”

Emergent Spaces for Coupled Oscillators

“…Finding these reduced sets of parameters often requires insight, along with trial and error. Here, we present a methodology for discovering such effective parameters in a data-driven way via a modification of diffusion maps, our manifold learning technique of choice in this paper (Holiday et al, 2019). A strong motivation for this work comes from the determination of explicit dimensionless parameters from the (possibly long) list of dimensional parameters of a physical model.…”