An exact viscosity solution to a Hamilton–Jacobi–Bellman quasi-variational inequality for animal population management

Yaegashi

2021

J.Math.Industry

Self Cite

A stochastic impulse control problem with imperfect controllability of interventions is formulated with an emphasis on applications to ecological and environmental management problems. The imperfectness comes from uncertainties with respect to the magnitude of interventions. Our model is based on a dynamic programming formalism to impulsively control a 1-D diffusion process of a geometric Brownian type. The imperfectness leads to a non-local operator different from the many conventional ones, and evokes a slightly different optimal intervention policy. We give viscosity characterizations of the Hamilton–Jacobi–Bellman Quasi-Variational Inequality (HJBQVI) governing the value function focusing on its numerical computation. Uniqueness and verification results of the HJBQVI are presented and a candidate exact solution is constructed. The HJBQVI is solved with the two different numerical methods, an ordinary differential equation (ODE) based method and a finite difference scheme, demonstrating their consistency. Furthermore, the resulting controlled dynamics are extensively analyzed focusing on a bird population management case from a statistical standpoint.

Section: Verificationmentioning

confidence: 83%

Mathematical and numerical analyses of a stochastic impulse control model with imperfect interventions

Yaegashi

2021

J.Math.Industry

Self Cite

“…Applying the classical divergence formula to (30) yields By (31), and taking the limit h → + 0, we infer…”

Section: Fokker-planck Equation and Pdfmentioning

confidence: 99%

“…Interventions that have much shorter time-scales than those of the target dynamics are reasonably considered to be impulsive [2]. Most of the impulse control models assume that the interventions are executed immediately at making the decisions [30,31], while there would be execution delays in real problems due to technical reasons like communication and implementation delays. So far, problems with the execution delay have been studied theoretically from classical [32,33] and viscosity viewpoints [34,35] with dynamic programming [36].…”

Section: Introductionmentioning

confidence: 99%

Stochastic impulse control of nonsmooth dynamics with partial observation and execution delay: Application to an environmental restoration problem

Optim Control Appl Methods

Yaegashi²

2021

Self Cite

Nonsmooth dynamics driven by stochastic disturbance arise in a wide variety of engineering problems. Impulsive interventions are often employed to control stochastic systems; however, the modeling and analysis subject to execution delay have been less explored. In addition, continuously receiving information of the dynamics is not always possible. In this article, with an application to an environmental restoration problem, a continuous-time stochastic impulse control problem subject to execution delay under discrete and random observations is newly formulated and analyzed. The dynamics have a nonsmooth coefficient modulated by a Markov chain, and eventually attain an undesirable state like a depletion due to the nonsmoothness. The goal of the control problem is to find the most cost-efficient policy that can prevent the dynamics from attaining the undesirable state. We demonstrate that finding the optimal policy reduces to solving a nonstandard system of degenerate elliptic equations, the Hamilton-Jacobi-Bellman equation (HJBE), which is rigorously and analytically verified in a simplified case. The associated Fokker-Planck equation (FPE) for the controlled dynamics is derived and solved explicitly as well. The model is finally applied to numerical computation of a recent river environmental restoration problem. The HJBE and FPE are successfully computed, and the optimal policy and the probability density functions are numerically obtained.The impacts of execution delay are discussed to deeper analyze the model.

“…x t are already found based on some optimality principle because our focus is the stochastic dynamics rather than the optimal policy. These thresholds can be found by solving the HJBQVI [1,2,3,22] associated with (1) and ( 3).…”

Section: Perfect Intervention Modelmentioning

confidence: 99%

Finite volume computation for the non-stationary probability density function of an impulsively controlled 1-D diffusion process

Yaegashi

Tsujimura

et al. 2020

JASSE

Self Cite

We derive a Fokker Planck Equation (FPE) governing probability density functions (PDFs) of an impulsively controlled 1-D diffusion process in seasonal population management problems. Two interventions are considered: perfect (completely controllable) and imperfect interventions (not completely controllable). The FPE is an initial-and boundary-value problem subject to a non-local boundary condition along a moving boundary. We show that an finite volume method (FVM) with a domain transformation realizes a conservative discretization for the FPE. We demonstrate that the computed PDFs with the FVM and those with a Monte Carlo method agree well.