Numerical method for optimal stopping of piecewise deterministic Markov processes

Abstract

We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location-inter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds for the convergence rate of the algorithm and to provide a computable ǫ-optimal stopping time. The paper is illustrated by a numerical example.

Optimal stopping for the predictive maintenance of a structure subject to corrosion

“…In reference [5], the authors propose a numerical method to approximate the value function for the optimal stopping problem of a PDMP. The approach is based on quantization of the post-jump locationinter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids.…”

“…. , t n(z) g defined by n(z) is the integer part minus 1 of t à (z)D À1 for 1 < i < n(z), t i = iD Grids are obtained that not only do not contain t à (z), but in addition, their maximum is strictly less than t à (z) À D, which is a crucial property to derive error bounds for the algorithm, see reference [5]. Note also that only a finite number of grids G(z) is needed, corresponding to the z in the quantization grids (G n ) 0 < n < N .…”

“…To get around the discontinuity of these functions, it is proved that the sets where they are actually discontinuous are of very low probability. The precise statement of this theorem and its proof can be found in reference [5].…”

“…This class of problems has been studied from a theoretical point of view in reference [5]. PDMPs are a class of stochastic hybrid processes that were introduced by Davis [6] in the 1980s.…”

A Stochastic Method based on the Markov Model of Unit Jump for Analyzing Crack Jump in a Material

“…Bo-Siou et al (2009) have proposed a model of Markov chains for predicting the evolution of damage, with a method for constructing a stochastic curve for a number of Composite Material Systems. Beil et al (2009), Brandejsky et al (2013), Chiquet et al (2009), Davis (1984, Davis (1993), Dhondt (1995), Rowatt et al (1998), de Saporta et al (2010), de Saporta et al (2012, Zhang et al (2014) have presented a Markov Model based on the evolution of damage to characterize fatigue behavior. Based on these behavioral studies, we investigated modeling the crack using a Markov Model with a method of simple analysis.…”