We consider the general problem of choosing a discounted cost minimizing infinite sequence of decisions from a closed subset of the product space formed by a sequence of arbitrary compact metric spaces. Examples include equipment replacement, production planning and, more generally, infinite stage mathematical programs. It is shown that the optimal costs for finite horizon approximating problems converge to the optimal infinite horizon cost as the horizons diverge to infinity. Moreover, the existence of a unique algorithmically optimal (i.e. accumulation point) solution is shown to be a necessary and sufficient condition for convergence in the product topology (i.e. policy convergence) of all finite horizon optima. Under the weaker condition of Hausdorff convergence of the sets of finite horizon optima to the set of infinite horizon optima, we show how to force policy convergence through a natural tie-breaking rule. Finally, a forward algorithm is presented which, in the presence of a unique infinite horizon optimum, is guaranteed to converge.

No abstract

We consider the problem of approximating an optimal solution to a separable, doubly infinite mathematical program (P) with lower staircase structure by solutions to the programs (P(N)) obtained by truncating after the first N variables and N constraints of (P). Viewing the surplus vector variable associated with the Nth constraint as a state, and assuming that all feasible states are eventually reachable from any feasible state, we show that the efficient set of all solutions optimal to all possible feasible surplus states for (P(N)) converges to the set of optimal solutions to (P). A tie-breaking algorithm which selects a nearest-point efficient solution for (P(N)) is shown (for convex programs) to converge to an optimal solution to (P). A stopping rule is provided for discovering a value of N sufficiently large to guarantee any prespecified level of accuracy. The theory is illustrated by an application to production planning.

No abstract

We consider the problem of solving a nonhomogeneous infinite horizon Markov Decision Process (MDP) problem in the general case of potentially multiple optimal first period policies. More precisely, we seek an algorithm that, given a finite subset of the problem's potentially infinite data set, delivers an optimal first period policy. Such an algorithm can thus recursively generate, within a rolling horizon procedure, an infinite horizon optimal solution to the original infinite horizon problem. However it can happen that for a given problem no such algorithm exists. In this case, it is impossible to solve the problem with a finite amount of data. We say such problems fail to be wellposed. Under the assumption of increasing marginal returns in actions (with respect to states) and stochastically increasing states into which the system transitions (with respect to actions), we provide an algorithm that is guaranteed to solve the corresponding nonhomogeneous MDP whenever the problem is well-posed. The algorithm proceeds by discovering, in finite time, a forecast horizon for which an optimal solution delivers an optimal first period policy to the infinite horizon problem. In particular, we show by construction, the existence of a forecast horizon (and hence, a solution horizon) for all such well-posed problems. We illustrate the theory and algorithms developed by solving every well-posed instance of the time-varying version of the classic asset selling problem.