Linear Programming Approach to Deterministic Infinite Horizon Optimal Control Problems with Discounting

Abstract

Abstract. We establish that deterministic long run average problems of optimal control are "asymptotically equivalent" to infinite-dimensional linear programming problems (LPPs) and we establish that these LPPs can be approximated by finite-dimensional LPPs, the solutions of which can be used for construction of the optimal controls. General results are illustrated with numerical examples.Key words. long run average optimal control, singularly perturbed control systems, occupational measures, averaging method, linear programming AMS subject classifications. 34E15, 34C29, 34A60, 93C70 DOI. 10.1137/0406168021. Introduction and description of the problems. In this paper we show that, under some conditions, deterministic long run average problems of optimal control are "asymptotically equivalent" to infinite-dimensional linear programming problems (LPPs) and we establish that these LPPs can be approximated by finitedimensional LPPs, the solutions of which can be used for numerical construction of the optimal controls.Infinite horizon problems of optimal control have been studied intensively in both deterministic and stochastic settings (see Anderson and Kokotovic [3] Mendiondo and Stockbridge [38]. A linear programming approach to long run average optimal control problems in the deterministic setting appears to be new and, to the best of our knowledge, there are no publications devoted to this topic (under different assumptions and for a different problem, a linear programming formulation was discussed in Evans and Gomes [20]). A linear programming approach to deterministic optimal control problems on a finite time interval has been studied in Rubio [42].Let us introduce the problems that we will be dealing with. Consider the control systemẏ

Linearization Techniques

“…The set Θ (t, r, x 0 ) is the closed convex hull of the family of occupational couples Γ (t, r, x 0 ) For further details, the reader is referred to [11]. …”

“…A wide literature is available on the subject both in the deterministic setting ( [11,12,17,22]) and in the stochastic framework ( [6][7][8][9]14]). Recently, the authors of [8,11,17] have provided linearized versions of the standard continuous infinite horizon discounted control problems. Their method is entirely based on Hamilton-Jacobi-Bellman equations and occupational measures.…”

On Average Control Generating Families for Singularly Perturbed Optimal Control Problems with Long Run Average Optimality Criteria

“…As in [48], our consideration is based on earlier results on averaging of SP control systems obtained in [39], [41], [42], [44], [47] (see also [6], [7], [8], [10], [51], [52], [67]) and on results obtained in [35], [45], [46], [47] that establish the equivalence of optimal control problems to certain infinite dimensional (ID) linear programming (LP) problems (related results on IDLP formulations of optimal control problems in both deterministic and stochastic settings can be found in [5], [15], [16], [21], [31], [37], [50], [53], [57], [60], [69], [71], [72] and [76]). In contrast to [48], where mostly optimal control problems with time discounting criteria were dealt with, this paper is devoted to the consideration of the problems with long run average optimality criteria.…”

Optimality Issues for a Class of Controlled Singularly Perturbed Stochastic Systems

“…[10], the study of the behavior of the value function is simpli…ed whenever this value is expressed by a linear problem. Recently, linearized versions of the standard continuous in…nite horizon discounted control problems have been provided in [9,11].…”

“…Following the methods presented in [8] and [9] for the deterministic controlled dynamics, one can approximate the occupational measures by Dirac measures and construct an optimal feedback control. Moreover, when considering the ergodic control problem, e.g.…”