A Low Complexity Approach to Model-Free Stochastic Inverse Linear Quadratic Control

Abstract

In this paper, we present a Model-Free Stochastic Inverse Optimal Control (IOC) algorithm for the discrete-time infinite-horizon stochastic linear quadratic regulator (LQR). Our proposed algorithm exploits the richness of the available system trajectories to recover the control gain K and cost function parameters (Q, R) in a low (space, sample, and computational) complexity manner. By leveraging insights on the stochastic LQR, we guarantee well-posedness of the Model-Free Stochastic IOC LQR via satisfaction of the Certainty Equivalence optimality conditions. The exact solution of the control gain K is recovered via a deterministic, low complexity Least Squares approach. Using K, we solve a completely model-free noniterative SemiDefinite Programming (SDP) problem to obtain a unique (up to a scalar ambiguity) (Q, R), in which optimality and feasibility are jointly ensured. Via derivation of the sample complexity bounds, we show that the non-asymptotic performance of the Model-Free Stochastic IOC LQR can be characterized by the signal-to-noise (SNR) ratio of the finite set of system state and input signals. We present a model-based version of the algorithm for the special case where (A, B) is available, and we, further, provide the extension to the Stochastic Model-Free IOC linear quadratic tracking (LQT) case.