Solving the Sailing Problem With a New Prioritized Value Iteration

Abstract

& In this paper we tackle the sailing strategies problem, a stochastic shortest-path Markov decision process. The problem of solving large Markov decision processes accurately and quickly is challenging. Because the computational effort incurred is considerable, current research focuses on finding superior acceleration techniques. For instance, the convergence properties of current solution methods depend, to a great extent, on the order of backup operations. On one hand, algorithms such as topological sorting are able to find good orderings, but their overhead is usually high. On the other hand, shortest path methods, such as Dijkstra's algorithm, which is based on priority queues, have been applied successfully to the solution of deterministic shortest-path Markov decision processes. Here, we propose improved value iteration algorithms based on Dijkstra's algorithm for solving shortest path Markov decision processes. The experimental results on a stochastic shortest-path problem show the feasibility of our approach.

Simplificación de los procesos de decisión de Markov mediante reglamentación de acciones y priorización de estados

“…Porúltimo, con el objeto de explorar otras estrategias de actualización de predecesores en el enfoque propuesto, se probaron las dos variantes del mismo [Garcia-Hernandez, 2012b] que se preentaron al final del Capítulo 4: JIPVI (IPVI con actualización de los predecesores de un estado prometedor mediante el método de Jacobi [Shlakhter, 2005], sin tener en cuenta su orden) y PPIPVI (IPVI con priorización de las actualizaciones de los predecesores de un estado prometedor).…”