Experimental Design for Partially Observed Markov Decision Processes

Abstract

sented. We discuss how parameter dependence within these methods can be dealt with by the use of priors, and develop tools to update control policies online. This is demonstrated in another stochastic dynamical system describing growth dynamics of DNA template in a PCR model.

Timing observations of diffusions

“…The second term, E y|(t,x) M i−1 (t, y), is the maximal information expected thereafter. Consequently we say that M i is the maximal Fisher information to go (FITG) for i observations, a phrase adopted from [1] and [6]. 1 Without loss of generality, we assume that, for all s and i, the supremum in (3) is achieved on the interval (s, τ ).…”

“…Discretizing Time: Often the δ needed to make the coefficients in (6) positive is so small that our policy scarcely changes across time increments of that size. For greater computational efficiency, we discretize time with the mesh…”

“…, t n ) and the expectation runs over all observations x other than the initial condition x 0 . The authors of [6] find that the value iteration algorithm, modified for their design problem, runs quickly enough to update π online. However, our paradigm of observation times poses greater computational burdens because, in our setting, the algorithm requires repeatedly multiplying P with itself.…”

“…We assume that sample paths of this system start from a known initial condition x 0 . By extending [1] and [6], we address the experimental design question:…”

“…We will see that our optimal policy depends on the very parameter, θ, that it is designed to estimate. Section 4 addresses this limitation by adapting the Bayesian machinery proposed by [1] and [6]. Section 5 concludes.…”