Optimal Approximation Schedules for a Class of Iterative Algorithms, With an Application to Multigrid Value Iteration

Adaptive Strategies for Accelerating the Convergence of Average Cost Markov Decision Processes Using a Moving Average Digital Filter

“…the optimal sequence with respect to the overall computational effort is geometrically decreasing, with rate  , which coincides with the convergence rate of the algorithm [13]. This result applies to discounted MDPs, for which the convergence rate  is known and coincides with the discount factor.…”

“…Hence, the sequence k  can be freely selected from the class of convergent sequences in the interval   0,1 whose limit is nil. However, it is the form at which the convergent sequence goes to zero that will ultimately determine the behavior and, therefore, the computational effort, of the PIVI algorithm [13,14].…”

“…This renders the direct application of the results in [13] unpractical. Indeed, geometrically decreasing sequences k  where tried in [14], and promising results where obtained.…”

“…These models are refined at each new iteration of the algorithm and converge to the exact model within a finite number of iterations, which enables one to retrieve the solution to the original problem at the end of the procedure. The rationale is based on a refinement scheme introduced in [13] for linearly convergent algorithms with convergence rates that are known a priori.…”

“…For more details on the convergence of VI algorithms for average cost MDPs, we refer to [1]. The unknown rate of convergence renders the results in [13] not directly applicable for the studied problem. Earlier results, however, have shown that significant reduction on the overall computational effort can be attained by a suitable choice of refinement rate [14].…”