Accelerated procedures for the solution of discrete Markov control problems

Kushner, Harold J.; Kleinman, Alan

doi:10.1109/tac.1971.1099667

Cited by 38 publications

(17 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The value of J* in this case is approximated by selecting a value between the two bounds. There are also several variations of the successive approximation method such as GaussSeidel iteration, successive over-relaxation [10], and Jacobi iteration [2]. Depending on the problem at hand these schemes may converge faster than the successive approximation method.…”

Section: Section 2: Successive Approximation and Error Boundsmentioning

confidence: 99%

Adaptive aggregation methods for infinite horizon dynamic programming

Bertsekas

Castañón

1989

IEEE Trans. Automat. Contr.

140

View full text Add to dashboard Cite

In this paper we propose an on-line policy iteration (PI) algorithm for finite-state infinite horizon discounted dynamic programming, whereby the policy improvement operation is done on-line, only for the states that are encountered during operation of the system. This allows the continuous updating/improvement of the current policy, thus resulting in a form of on-line PI that incorporates the improved controls into the current policy as new states and controls are generated. The algorithm converges in a finite number of stages to a type of locally optimal policy, and suggests the possibility of variants of PI and multiagent PI where the policy improvement is simplified. Moreover, the algorithm can be used with on-line replanning, and is also well-suited for on-line PI algorithms with value and policy approximations.

show abstract

Section: Section 2: Successive Approximation and Error Boundsmentioning

confidence: 99%

Adaptive aggregation methods for infinite horizon dynamic programming

Bertsekas

Castañón

1989

IEEE Trans. Automat. Contr.

140

View full text Add to dashboard Cite

show abstract

“…Most of these variations achieve the same rate of convergence as the techniques above. In addition, [2] mentions other iterative schemes such as Gauss -Seidel iteration, successive over-relaxation [10], and Jacobi iteration. For specific problems, the convergence rate of any one of these techniques may be better than the rate of the successive approximation iteration of eq.…”

Section: -Amentioning

confidence: 99%

Adaptive aggregation methods for discounted dynamic programming

Bertsekas

Castañón

1986

1986 25th IEEE Conference on Decision and Control

View full text Add to dashboard Cite

“…The approximating chains provide a very convenient computational tool for obtaining approximations to various functionals of diffusions, since there are numerous efficient computational techniques for Markov chain problems (see, e.g. Kushner and Kleinman [10]). The same type of approximations will be used here.…”

Section: Formulation Of the Impulsive Control Problemmentioning

confidence: 99%

Approximations and computational methods for Optimal Stopping and Stochastic Impulsive Control problems

Kushner

1976

Appl Math Optim

View full text Add to dashboard Cite

The paper treats a computational method for the Optimal Stopping and Stochastic Impulsive Control problem for a diffusion. In the latter problem control acts only intermittently since there is a basic positive "transaction" cost to be paid at each instant that the control acts. For each h > 0, a controlled Markov chain is constructed, whose continuous time interpolations are a natural approximation to the diffusion, for both the optimal stopping and impulsive control situations. The solutions to the optimal stopping and impulsive control problems for the chains are relatively easy to obtain by using standard procedures, and they converge to the solutions of the corresponding problems for the diffusion models as h-,0.

show abstract

Accelerated procedures for the solution of discrete Markov control problems

Abstract: Accelerated procedures for computing optimal controls for a Markov chain model are discussed and numerical resdts are presented. For the example, one of the methods described gave a ten-fold decrease io computation time over a more usnal procednre of dynamic programming.

Cited by 38 publications

References 1 publication

Adaptive aggregation methods for infinite horizon dynamic programming

Adaptive aggregation methods for infinite horizon dynamic programming

Adaptive aggregation methods for discounted dynamic programming

Approximations and computational methods for Optimal Stopping and Stochastic Impulsive Control problems

Contact Info

Product

Resources

About