Approximate dynamic programming for stochastic linear control problems on compact state spaces

Woerner, Stefan; Laumanns, Marco; Zenklusen, Rico; Fertis, Apostolos G.

doi:10.1016/j.ejor.2014.08.003

Cited by 10 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Calculations obtained with the help of the algorithm from subsection 2.3. 9. r 5 10. r 10 11. r 16 12. r 6 13. r 15 14. r 9 15. r 4 16. r 12…”

Section: E Compromise Hypersphere Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Compromise Hypersphere for Multi-Criteria Dynamic Programming

Sitarz

2015

IJIEEB

View full text Add to dashboard Cite

The paper considers multi-criteria dynamic decision process. We focus on the efficient realizations of the dynamic process which are characterized by nondominated values of the multi-period criteria function. The aim of the paper is to use the compromise hypersphere method to rank the efficient realizations. The presented method allows us to take into account the risk aversion of the decision maker. Moreover, we apply the presented theory in the market model taken from microeconomic theory. Index Terms-Dynamic programming, multi-criteria decision making, compromise hypersphere.

show abstract

“…Calculations obtained with the help of the algorithm from subsection 2.3. 9. r 5 10. r 10 11. r 16 12. r 6 13. r 15 14. r 9 15. r 4 16. r 12…”

Section: E Compromise Hypersphere Methodsmentioning

confidence: 99%

“…Sitarz [4] shows a usage of dynamic programming into multiple knapsack problem. In [5] the stochastic dynamic programming on compact state spaces is discussed. Whereas the idea of compromise hypersphere comes from the work by Gass and Roy [6].…”

Section: Introductionmentioning

confidence: 99%

Compromise Hypersphere for Multi-Criteria Dynamic Programming

Sitarz

2015

IJIEEB

View full text Add to dashboard Cite

show abstract

“…However, state space, action space and external information space grow as the size of the problem is getting larger, causing the curse of dimensionality. Approximate dynamic programming [17] take advantages of the optimal control theory, approximation techniques and reinforcement learning theory, mainly aiming at solving dimensionality curse problems. The main idea of approximate dynamic programming is to use approximate method to predict and control the state of the approximate value function by Monte Carlo [18] simulation and statistical methods to sample states and external information…”

Section: Approximate Dynamic Programmingmentioning

confidence: 99%

Sarsa(Λ)-Based Logistics Planning Approximated by Value Function with Policy Iteration

Yong

2015

Journal of Algorithms & Computational Technology

View full text Add to dashboard Cite

The logistics planning problem has been extensively investigated for a long time. However, with the increasing number of stochastic events occurred in road, increasing number of stochastic factors should be taken into consideration. A dynamic approach is used in this paper to solve the logistics planning problem in the common form of stochastic demand with the reinforcement learning framework which is able to optimize policy in unknown environments and uncertain cases. We take advantage of clustering method to extract states as main features for basis function so as to solve the dimensionality curse problems caused by stochastic settings. We also propose an approximation approach with the policy iteration restricted by the goal of minimal time differential error to approximate the stochastic cases of the real world, and then use the attained approximation parameters as input for the proposed Sarsa(λ)-based logistics planning algorithm to determine the policy and action in accordance with the real world stochastic events. The benchmarking experimental results showed that the proposed algorithm has achieved improvements in almost all the test cases.

show abstract

“…Beyond Chow, there are of course many other important contributions to the topic of stochastic optimal control in continuous time, many of these use duality, for example Ma et al (2020), and it would be very interesting to connect the Lagrange formalism explored in our paper to these. Further, Wörner et al (2015) propose an approximate relative value iteration algorithm based on piecewise-linear convex relative value function approximations, which would be interesting to investigate from the perspective of the Lagrange multiplier theorem and our proposed numerical scheme in section 4, which is partly Monte Carlo based. We leave both of these avenues for future research.…”

Section: Introductionmentioning

confidence: 99%

On the Adaptation of the Lagrange Formalism to Continuous Time Stochastic Optimal Control: A Lagrange-Chow Redux

Ewald,

Nolan

2023

Preprint

View full text Add to dashboard Cite

Approximate dynamic programming for stochastic linear control problems on compact state spaces

Cited by 10 publications

References 27 publications

Compromise Hypersphere for Multi-Criteria Dynamic Programming

Compromise Hypersphere for Multi-Criteria Dynamic Programming

Sarsa(Λ)-Based Logistics Planning Approximated by Value Function with Policy Iteration

On the Adaptation of the Lagrange Formalism to Continuous Time Stochastic Optimal Control: A Lagrange-Chow Redux

Contact Info

Product

Resources

About