System Identification of Static Nonlinear Elements: A Unified Approach of Active Learning, Over-fit Avoidance, and Model Structure Determination

Abstract

Systems containing linear first-order dynamics and static nonlinear elements (i.e., nonlinear elements whose outputs depend only on the present value of inputs) are often encountered; for example, certain automobile engine subsystems. Therefore, system identification of static nonlinear elements becomes a crucial component that underpins the success of the overall identification of such dynamical systems. In relation to identifying such systems, we are often required to identify models in differential equation form, and consequently, we are required to describe static nonlinear elements in the form of functions in time domain. Identification of such functions describing static elements is often a black-box identification exercise; although the inputs and outputs are known, correct mathematical models describing the static nonlinear elements may be unknown. Therefore, with the aim of obtaining computationally efficient models, calibrating polynomial models for such static elements is often attempted. With that approach comes several issues, such as long time requirements to collect adequate amounts of measurements to calibrate models, having to test different models to pick the best one, and having to avoid models over-fitting to noisy measurements. Given that premise, this paper proposes an approach to tackle some of those issues. The approach involves collecting measurements based on an uncertainty-driven Active Learning scheme to reduce time spent on measurements, and simultaneously fitting smooth models using Gaussian Process (GP) regression to avoid over-fitting, and subsequently picking best fitting polynomial models using GP-regressed smooth models. The principles for the single-input-single-output (SISO) static nonlinear element case are demonstrated in this paper through simulation. These principles can easily be extended to MISO systems.

D-Optimal Design for Information Driven Identification of Static Nonlinear Elements

“…To account for noise, authors propose each measurement in a batch of p being repeated s, s ∈ Z + times, and the mean of the s repetitions being taken as a single measurement as done in [9].…”

“…Previous work attempting information maximization focused efficient measurement taking have often relied on Fisher Information [4], [5] driven design of experiments [1], [4], [6], and so-called Active Learning strategies that have adopted techniques like 'query by committee' [7] and Gaussian Process (GP) [8] based uncertainty reduction [9], [10]. From such approaches, some quite powerful ones are the in-the-loop information maximization methods [9].…”

“…Previous work attempting information maximization focused efficient measurement taking have often relied on Fisher Information [4], [5] driven design of experiments [1], [4], [6], and so-called Active Learning strategies that have adopted techniques like 'query by committee' [7] and Gaussian Process (GP) [8] based uncertainty reduction [9], [10]. From such approaches, some quite powerful ones are the in-the-loop information maximization methods [9]. What such methods often do, is for any given prior set of measurements, they calculate the 'next best measurement' via information maximization in-the-loop with simultaneous measurement-taking and model calibration [9].…”

“…From such approaches, some quite powerful ones are the in-the-loop information maximization methods [9]. What such methods often do, is for any given prior set of measurements, they calculate the 'next best measurement' via information maximization in-the-loop with simultaneous measurement-taking and model calibration [9]. Although such methods have shown to accomplish the task of optimizing measurement-taking while identification of adequate models, they can add significant computation and real-time data interfacing challenges for the measurement-taking process [9].…”

“…What such methods often do, is for any given prior set of measurements, they calculate the 'next best measurement' via information maximization in-the-loop with simultaneous measurement-taking and model calibration [9]. Although such methods have shown to accomplish the task of optimizing measurement-taking while identification of adequate models, they can add significant computation and real-time data interfacing challenges for the measurement-taking process [9]. The reason behind computation cost of such methods attribute to factors like the use of techniques like GP, or more generally, taking leads from Bayesian Optimization.…”