Dynamic importance sampling for uniformly recurrent Markov chains

Abstract

Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested by a large deviation lower bound analysis. Recent work, however, has suggested that such schemes do not work well in many situations. In this paper we consider dynamic importance sampling in the setting of uniformly recurrent Markov chains. By "dynamic" we mean that in the course of a single simulation, the change of measure can depend on the outcome of the simulation up till that time. Based on a control-theoretic approach to large deviations, the existence of asymptotically optimal dynamic schemes is demonstrated in great generality. The implementation of the dynamic schemes is carried out with the help of a limiting Bellman equation. Numerical examples are presented to contrast the dynamic and standard schemes.

Importance Sampling of Word Patterns in DNA and Protein Sequences

“…0 (Sadowsky and Bucklew, 1990;Dupuis and Wang, 2005) The examples described in Sections 3-5 involve word families that can be characterized as V m . We may also include an additional subscript m to a previously defined quantity to highlight its dependence on m, for example p m , q m , b m and n m .…”

Asymptotically optimal importance sampling for Jackson networks with a tree topology

“…To construct our optimal IS algorithms we use an optimality result from [16] which was obtained using the optimal control/subsolution approach to IS of [12,3,4,6,5]. This result states that to construct optimal IS algorithms for the simulation of a wide range of buffer overflow events of any stable Jackson network it is sufficient to build appropriate smooth subsolutions to a Hamilton Jacobi Bellman (HJB) equation and its boundary conditions (these are given in (7) in the context we study in the current paper).…”

Subsolutions of an Isaacs Equation and Efficient Schemes for Importance Sampling: Examples and Numerics

“…It was established in [4,5] that importance sampling algorithms for estimating rare-event probabilities or functionals that are largely determined by rare-events are closely related to deterministic differential games. More precisely, the asymptotic optimal performance of importance sampling schemes can be characterized by the value function of a two-person zero-sum differential game, which can in turn be characterized by the solution to the Isaacs equation (a nonlinear PDE) associated with the game.…”

“…More precisely, the asymptotic optimal performance of importance sampling schemes can be characterized by the value function of a two-person zero-sum differential game, which can in turn be characterized by the solution to the Isaacs equation (a nonlinear PDE) associated with the game. It was also discussed in [4,5] that one can construct asymptotically optimal importance sampling algorithms based on this solution.…”

“…

AbstractIt was established in [4,5] that importance sampling algorithms for estimating rare-event probabilities are intimately connected with twoperson zero-sum differential games and the associated Isaacs equation. The purpose of the present paper and a companion paper [6] is to show that the classical sense subsolutions of the Isaacs equation can be used as a basic and flexible tool for the construction and analysis of efficient importance sampling schemes.

…”