Active Learning of Dynamics for Data-Driven Control Using Koopman Operators

Abraham, Ian; Murphey, Todd D.

doi:10.1109/tro.2019.2923880

Cited by 141 publications

(94 citation statements)

References 58 publications

(128 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This includes demonstrations that show the Koopman operator can differentiate between cyclic and non-cyclic stochastic signals in stock market data (Hua et al, 2016) and that it can detect specific signals in neural data that signify non-rapid eye movement (NREM) sleep (Brunton et al, 2016). More recently these systems have included physical robotics systems (Abraham and Murphey, 2019; Bruder et al, 2019).…”

Section: Background and Related Workmentioning

confidence: 99%

Data-driven Koopman operators for model-based shared control of human–machine systems

Broad

Abraham

Murphey

et al. 2020

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

We present a data-driven shared control algorithm that can be used to improve a human operator’s control of complex dynamic machines and achieve tasks that would otherwise be challenging, or impossible, for the user on their own. Our method assumes no a priori knowledge of the system dynamics. Instead, both the dynamics and information about the user’s interaction are learned from observation through the use of a Koopman operator. Using the learned model, we define an optimization problem to compute the autonomous partner’s control policy. Finally, we dynamically allocate control authority to each partner based on a comparison of the user input and the autonomously generated control. We refer to this idea as model-based shared control (MbSC). We evaluate the efficacy of our approach with two human subjects studies consisting of 32 total participants (16 subjects in each study). The first study imposes a linear constraint on the modeling and autonomous policy generation algorithms. The second study explores the more general, nonlinear variant. Overall, we find that MbSC significantly improves task and control metrics when compared with a natural learning, or user only, control paradigm. Our experiments suggest that models learned via the Koopman operator generalize across users, indicating that it is not necessary to collect data from each individual user before providing assistance with MbSC. We also demonstrate the data efficiency of MbSC and, consequently, its usefulness in online learning paradigms. Finally, we find that the nonlinear variant has a greater impact on a user’s ability to successfully achieve a defined task than the linear variant.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Data-driven Koopman operators for model-based shared control of human–machine systems

Broad

Abraham

Murphey

et al. 2020

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further [3] shows the convergence in the strong operator topology of the Koopman operator computed via EDMD [4] to the actual Koopman operator as the number of data points and the number of observables tend to infinity. Koopman operator theory (KOT) have been applied to robotics applications [5,6,7,8,9], power grid stabilization [10,11], state estimation [12,13], control synthesis [14,15,16,17], actuator and sensor placement [18], aerospace applications [19,20], analysis of climate, fluid mechanics, human-machine systems [21], chemical process systems [22] and control of PDEs. Koopman operator theory postulates that a nonlinear uncontrolled system can be lifted to an equivalent (infinite-dimensional) linear system whereas a nonlinear control system to a bilinear control system.…”

Section: Introductionmentioning

confidence: 99%

Neural Koopman Lyapunov Control

Zinage¹,

Bakolas²

2022

Preprint

View full text Add to dashboard Cite

Learning and synthesizing stabilizing controllers for unknown nonlinear systems is a challenging problem for real-world and industrial applications. Koopman operator theory allow one to analyze nonlinear systems through the lens of linear systems and nonlinear control systems through the lens of bilinear control systems. The key idea of these methods, lies in the transformation of the coordinates of the nonlinear system into the Koopman observables, which are coordinates that allow the representation of the original system (control system) as a higher dimensional linear (bilinear control) system. However, for nonlinear control systems, the bilinear control model obtained by applying Koopman operator based learning methods is not necessarily stabilizable and therefore, the existence of a stabilizing feedback control is not guaranteed which is crucial for many real world applications. Simultaneous identification of these stabilizable Koopman based bilinear control systems as well as the associated Koopman observables is still an open problem. In this paper, we propose a framework to identify and construct these stabilizable bilinear models and its associated observables from data by simultaneously learning a bilinear Koopman embedding for the underlying unknown nonlinear control system as well as a Control Lyapunov Function (CLF) for the Koopman based bilinear model using a learner and falsifier. Our proposed approach thereby provides provable guarantees of global asymptotic stability for the nonlinear control systems with unknown dynamics. Numerical simulations are provided to validate the efficacy of our proposed class of stabilizing feedback controllers for unknown nonlinear systems.

show abstract

“…The dynamic mode decomposition (DMD) is a data diagnostic technique that extracts coherent spatial-temporal patterns from high-dimensional time series data [19,24]. Although DMD originated in the fluid dynamics community [19], the algorithm has since been applied to a wealth of dynamical systems including in epidemiology [25], robotics [26,27], neuroscience [28], quantum control [29], power grids [30], and plasma physics [31,32]. Despite its widespread successes, DMD is highly sensitive to noise [33][34][35], fails to capture travelling wave physics, and can produce overfit models that do not generalize.…”

Section: Introductionmentioning

confidence: 99%

Physics-informed dynamic mode decomposition (piDMD)

Baddoo¹,

Herrmann²,

McKeon³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work, we demonstrate how physical principles -such as symmetries, invariances, and conservation laws -can be integrated into the dynamic mode decomposition (DMD). DMD is a widely-used data analysis technique that extracts low-rank modal structures and dynamics from high-dimensional measurements. However, DMD frequently produces models that are sensitive to noise, fail to generalize outside the training data, and violate basic physical laws. Our physics-informed DMD (piDMD) optimization, which may be formulated as a Procrustes problem, restricts the family of admissible models to a matrix manifold that respects the physical structure of the system. We focus on five fundamental physical principles -conservation, self-adjointness, localization, causality, and shift-invariance -and derive several closed-form solutions and efficient algorithms for the corresponding piDMD optimizations. With fewer degrees of freedom, piDMD models are less prone to overfitting, require less training data, and are often less computationally expensive to build than standard DMD models. We demonstrate piDMD on a range of challenging problems in the physical sciences, including energypreserving fluid flow, travelling-wave systems, the Schr ödinger equation, solute advectiondiffusion, a system with causal dynamics, and three-dimensional transitional channel flow. In each case, piDMD significantly outperforms standard DMD in metrics such as spectral identification, state prediction, and estimation of optimal forcings and responses.

show abstract

Active Learning of Dynamics for Data-Driven Control Using Koopman Operators

Cited by 141 publications

References 58 publications

Data-driven Koopman operators for model-based shared control of human–machine systems

Data-driven Koopman operators for model-based shared control of human–machine systems

Neural Koopman Lyapunov Control

Physics-informed dynamic mode decomposition (piDMD)

Contact Info

Product

Resources

About