No abstract

For systems with only known pixels, it is difficult to identify its dynamics, especially with a linear operator. In this work, we present a convolutional neural network (CNN) based on the Koopman operator (CKNet) to identify the latent dynamics from raw pixels. CKNet learned an encoder and decoder to play the role of the Koopman eigenfunctions and modes, respectively. The Koopman eigenvalues can be approximated by the eigenvalues of the learned system matrix. We present the deterministic and variational approaches to realize the encoder separately. Because CKNet is trained under the constraints of the Koopman theory, the identified dynamics is linear, controllable and physically-interpretable. Besides, the system matrix and control matrix are trained as trainable tensors. To improve the performance, we propose the auxiliary weight term for multistep linearity and prediction losses. Experiments select two classic forced dynamical systems with continuous action space, and the results show that identified dynamics with 32-dim can predict validly 120 steps and generate clear images.

Autonomous driving has attracted lots of attention in recent years. An accurate vehicle dynamics is important for autonomous driving techniques, e.g. trajectory prediction, motion planning, and control of trajectory tracking. Although previous works have made some results, the strong nonlinearity, precision, and interpretability of dynamics for autonomous vehicles are open problems worth being studied. In this paper, the approach based on the Koopman operator named deep direct Koopman (DDK) is proposed to identify the model of the autonomous vehicle and the identified model is a linear time-invariant (LTI) version, which is convenient for motion planning and controller design. In the approach, the Koopman eigenvalues and system matrix are considered as trainable tensors with the original states of the autonomous vehicle being concatenated to a part of the Koopman eigenfunctions so that a physically interpretable subsystem can be extracted from the identified latent dynamics. Subsequently, the process of the identification model is trained under the proposed method based on the dataset which consists of about 60km of data collected with a real electric SUV while the effectiveness of the identified model is validated. Meanwhile, a high-fidelity vehicle dynamics is identified in CarSim with DDK, and then, a linear model predictive control (MPC) called DDK-MPC integrating DDK is designed to validate the performance for the control of trajectory tracking. Simulation results illustrate that the model of the nonlinear vehicle dynamics can be identified effectively via the proposed method and that excellent tracking performance can be obtained with the identified model under DDK-MPC.

Although previous studies have made some clear leap in learning latent dynamics from high‐dimensional representations, the performances in terms of accuracy and inference time of long‐term model prediction still need to be improved. In this study, a deep convolutional network based on the Koopman operator (CKNet) is proposed to model non‐linear systems with pixel‐level measurements for long‐term prediction. CKNet adopts an autoencoder network architecture, consisting of an encoder to generate latent states and a linear dynamical model (i.e., the Koopman operator) which evolves in the latent state space spanned by the encoder. The decoder is used to recover images from latent states. According to a multi‐step ahead prediction loss function, the system matrices for approximating the Koopman operator are trained synchronously with the autoencoder in a mini‐batch manner. In this manner, gradients can be synchronously transmitted to both the system matrices and the autoencoder to help the encoder self‐adaptively tune the latent state space in the training process, and the resulting model is time‐invariant in the latent space. Therefore, the proposed CKNet has the advantages of less inference time and high accuracy for long‐term prediction. Experiments are performed on OpenAI Gym and Mujoco environments, including two and four non‐linear forced dynamical systems with continuous action spaces. The experimental results show that CKNet has strong long‐term prediction capabilities with sufficient precision.

No abstract