A Shadow Simplex Method for Infinite Linear Programs

Abstract

We present a Simplex-type algorithm, that is, an algorithm that moves from one extreme point of the infinite-dimensional feasible region to another not necessarily adjacent extreme point, for solving a class of linear programs with countably infinite variables and constraints. Each iteration of this method can be implemented in finite time, while the solution values converge to the optimal value as the number of iterations increases. This Simplex-type algorithm moves to an adjacent extreme point and hence reduces to a true infinite-dimensional Simplex method for the important special cases of non-stationary infinite-horizon deterministic and stochastic dynamic programs.

A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes

“…It is easy to show (see for example Proposition 2.7 in [22]) that the objective function in (D) is continuous, and the feasible region is nonempty and compact. Hence it has an optimal solution, justifying our use of min instead of inf.…”

“…Recall from Section 2 that (D) has an optimal solution. In fact, since the feasible region of (D) is convex and the objective function is linear, Bauer's Maximum Principle [5] implies that (D) has an extreme point optimal solution (also see Proposition 2.7 in [22]). Also recall from Section 2 that, in any feasible solution x to (D), x n (s, a) > 0 for at least one a ∈ A for each n ∈ N and s ∈ S. We then have Definition 4.2.…”

“…The algorithm reaches an optimal extreme point after a finite number of iterations and then stops. The difficulties in replicating this in the context of CILPs, even when duality results and basic feasible solution characterization of extreme points are available, have been outlined in [6,22,44]. Even checking feasibility of a given solution may in general require infinite data and computations.…”

“…A pivot operation may require infinite computations and hence may not be implementable [6,44]. Finally, a sequence of CILP extreme points with strictly improving objective function values may not converge in value to optimal [22]. Noticing such pathologies, Anderson and Nash commented on page 73 in their seminal book [6], "... any algorithm will be difficult to implement; it is hard even to check feasibility."…”

“…However, since the CILP formulation of nonstationary MDPs does not belong to this class, their approach is not applicable here. Ghate et al [22] presented a Simplex-type method for a larger class of CILPs that subsumes nonstationary MDPs. That algorithm however was akin to a planning horizon approach and even though it produced a sequence of adjacent extreme points, it did not utilize duality and basic feasible solution characterization of extreme points, and in particular did not guarantee that the sequence was improving in objective values.…”