Analytical and numerical solutions to ergodic control problems arising in environmental management

Abstract

Environmental management optimizing a long-run objective is an ergodic control problem whose resolution can be achieved by solving an associated nonlocal Hamilton-Jacobi-Bellman (HJB) equation having an effective Hamiltonian. Focusing on sediment storage management as a modern engineering problem, we formulate, analyze, and compute a new ergodic control problem under discrete observations: a simple but non-trivial mathematical problem. We give optimality and comparison results of the corresponding HJB equation having unique non-smoothness and discontinuity. To numerically compute HJB equations, we propose a new fast-sweep method resorting to neither pseudo-time integration nor vanishing discount. The optimal policy and the effective Hamiltonian are then computed simultaneously. Convergence rate of numerical solutions is computationally analyzed. An advanced robust control counterpart where the dynamics involve uncertainties is also numerically considered.

Time‐average stochastic control based on a singular local Lévy model for environmental project planning under habit formation

“…Convergence rate an H is not computable by the previous method tailored for constant H [34]. The obtained numerical solutions are bounded above by 1 from above, the theoretical upper bound of H, and do not contain spurious oscillations.…”

“…Proposition 2. The SDE (34) with an initial condition X 0 D admits a unique strong solution that is càdlàg and bounded in D almost surely.…”

“…Impulses as opportunities to replenish the sediment are assumed to arrive as jump times of a Poisson process. The Poisson assumption is key to deriving tractable stochastic control models with discrete-time observations [44,45], which is a reasonable assumption as the time scale of replenishment activities is much shorter than that of the sediment decrease, and further continuous observation of sediment storage is difficult in general [33,34]. Subsequently, the Hamilton-Jacobi-Bellman (HJB) equation of this control problem and its discounted version also referred to as Abel limit [46] are formally obtained.…”

“…Remark 1. The stable subordinator is used to formulate a minimal stochastic model that can handle a diversity of jump patterns [34].…”

“…The ergodic control has been studied in a variety of problems, including but not limited to resource harvesting [28], optimization of noisy switching dynamics [29], planning of risk shearing pension design [30], queue servers optimization [31], and multi-agent optimization [32]. To realize water environmental management, the authors considered ergodic control problems with phase-type intervention schemes [33,34]. However, the ergodicity assumption sometimes fails for pathdependent control problems including those subjected to habit formation.…”