A Dynamic Program for a Team of Two Agents with Nested Information

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

“…From Assumption 2, the observation and cost functions at each t are Lipschitz. Using these properties in ( 12) -( 13), we can show that the value functions are continuous and the conditional ranges are compact for all t, which implies that the minimum is achieved in the RHS of (11). Thus, compactness of all marginal ranges and Assumptions 1 -2 constitute sufficient conditions for existence of an optimal solution to Problem 1, which is consistent with the conditions given in [90].…”

“…, T , then this DP yields a control law at time t as ḡ * t (π t ) := arg min ut∈[[Ut]] Qt (π t , u t ). Next, we prove that the DP ( 16) -( 17) computes the same value as the optimal DP (10) - (11).…”

“…This establishes that the second term in the RHS of (10) equals the second term in the RHS of ( 16) and subsequently, that given the induction hypothesis for time t + 1, we have Q t (m t , u t ) = Qt (σ t (m t ), u t ). Next, we minimize both sides of the equality with respect to u t ∈ [[U t ]], and use the definitions of the value functions in (11) and (17)…”

“…Next, we establish an upper bound on the approximation error when the value functions of the optimal DP (10) - (11) are estimated using the approximate DP ( 25) -( 26) at each t. Theorem 3. Let L Vt+1 be the Lipschitz constant of Vt+1 for all t = 0, .…”

“…Then, the performance of the memory-based approximate strategy g ap is Λ 0 (m 0 ). In contrast, recall that the performance of an optimal strategy g * is the optimal value V 0 (m 0 ) computed using ( 10) - (11). Next, we bound the difference in performance between g ap and g * .…”