Online Observer-Based Inverse Reinforcement Learning

Abstract

In this paper, a novel approach to the outputfeedback inverse reinforcement learning (IRL) problem is developed by casting the IRL problem, for linear systems with quadratic cost functions, as a state estimation problem. Two observer-based techniques for IRL are developed, including a novel observer method that re-uses previous state estimates via history stacks. Theoretical guarantees for convergence and robustness are established under appropriate excitation conditions. Simulations demonstrate the performance of the developed observers and filters under noisy and noise-free measurements.

Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games

“…Proof: Bring such Q * , R, γ into (19a), since Q T , R T , γ T satisfy (6), then one has P * = cP T and K * = K T . Then, using this result in (11) for V (x) * and comparing it with V T (x T ) * in (4) shows that V (x) * = cV T (x T ) * .…”

“…Lemma 1. Consider the two-player learner agent (7) with x(t 0 ) = x 0 , t ≥ t 0 and ( 8) and (11). Assume there exists a positive definite symmetric matrix P ∈ R n×n such that…”

“…corollary 1. With scalar c > 0, any 4) and (11), and they optimally associate with the same K * as the expert such that K * = K T in (19b) and (3a).…”