Simulation-based Algorithms for Markov Decision Processes

Abstract

British Library Cataloguing in Publication Data Simulation-based algorithms for Markov decision processes. -(Communications and control engineering) 1. Decision making -Mathematical models 2. Markov processes I. Chang, Hyeong Soo engineering -Mathematics 3. Matrices 4. Linear systems 658.4'033 ISBN-13: 9781846286896

Random search for constrained Markov decision processes with multi-policy improvement

“…Even if the spirit of improving all policies in a set is similar to parallel rollout in unconstrained MDPs (Chang, Hu, Fu, & Marcus, 2013), the approach here is different due to the constrained setting. The parallel rollout method for a set of policies uses the maximum value function over the value functions of the policies in the set at each possible next state when it obtains an action prescribed by an improving policy at a state.…”

One-stage R&D portfolio optimization with an application to solid oxide fuel cells

“…Reference [37] gives a survey of EC applied to noisy environments. Recent work by [13] and [12] has produced provably convergent algorithms for solving Markov Decision Processes. Reference [32] extend this work to solving problems with the form of (1).…”

“…Evolutionary Policy Iteration (EPI) was proposed in [13] and [12]. It was suggested as a method to find optimal policies in Markov decision processes (MDP).…”

“…Reference [12] notes that E C(ρ 1 (x)) could be computed via simulation, but does not offer a sampling routine or convergence results. Since E C(ρ 1 (x)) can only be approximated via Monte Carlo simulations in this application, this section will provide a modification of Algorithm 2 to handle this situation and provides a proof of convergence for the resulting algorithm.…”

“…Both proposals of EPI [12,13] assume that E C(ρ 1 (x)) can be computed without error for every portfolio x. Reference [12] notes that E C(ρ 1 (x)) could be computed via simulation, but does not offer a sampling routine or convergence results.…”