Solving Markov decision processes for network-level post-hazard recovery via simulation optimization and rollout

Abstract

Computation of optimal recovery decisions for community resilience assurance post-hazard is a combinatorial decision-making problem under uncertainty. It involves solving a large-scale optimization problem, which is significantly aggravated by the introduction of uncertainty. In this paper, we draw upon established tools from multiple research communities to provide an effective solution to this challenging problem. We provide a stochastic model of damage to the water network (WN) within a testbed community following a severe earthquake and compute near-optimal recovery actions for restoration of the water network. We formulate this stochastic decisionmaking problem as a Markov Decision Process (MDP), and solve it using a popular class of heuristic algorithms known as rollout. A simulation-based representation of MDPs is utilized in conjunction with rollout and the Optimal Computing Budget Allocation (OCBA) algorithm to address the resulting stochastic simulation optimization problem. Our method employs nonmyopic planning with efficient use of simulation budget. We show, through simulation results, that rollout fused with OCBA performs competitively with respect to rollout with total equal allocation (TEA) at a meagre simulation budget of 5-10% of rollout with TEA, which is a crucial step towards addressing large-scale community recovery problems following natural disasters.Saeed.Nozhati, Bruce.Ellingwood,

Optimal stochastic dynamic scheduling for managing community recovery from natural hazards

“…We use the Abrahamson et al [21] Ground Motion Prediction Equation (GMPE) to estimate the median seismic demands (Intensity Measures) on infrastructure facilities: Peak Ground Acceleration (PGA) for the EPN components and above-ground WN facilities and Peak Ground Velocity (PGV) for buried pipelines. We assess the physical damage to components with seismic fragility curves [22][23][24][25].…”

A novel simulation-annealing enabled ranking and scaling statistical simulation constrained optimization algorithm for Internet-of-things (IoTs)

“…Sarkale et al (2018) proposed an optimal recovery decision-based combinatorial decision-making challenge for critical infrastructure. This work compiles a list of tools helpful in solving critical infrastructure-based challenges.…”

Near-optimal planning using approximate dynamic programming to enhance post-hazard community resilience management

“…The policy iteration algorithm (see [44] for the details of the policy iteration algorithm including the definition of policy in the dynamic programming sense) computes an improved policy (policy improvement step), given a base policy (stationary), by evaluating the performance of the base policy. The policy evaluation step is typically performed through simulations [37]. Rollout policy can be viewed as the improved policy calculated using the policy iteration algorithm after a single iteration of the policy improvement step.…”

“…We are preparing a separate study to address the relaxation of this assumption, i.e., when outcomes of decisions exhibit uncertainty. The modified methods to deal with the stochastic problem will form a part of a separate paper [37].…”