Long-term adaption decisions via fully and partially observable Markov decision processes

Abstract

Long term decisions, such as the design of infrastructure systems and buildings or the planning of risk mitigation measures, should be made in consideration of the uncertain future. The initial design of a system determines its flexibility, i.e. its ability to cope with potential future changes. Increasing flexibility is generally considered to be a good approach to dealing with future uncertainty, such as climate change uncertainty, but its effects have not been systematically investigated. We propose the use of Markov Decision Processes combined with Influence Diagrams to solve adaptation planning problems. This framework can identify the optimal system type and capacity and determine the value of flexibility. It is here applied to two numerical examples: Planning of a waste water treatment plant under uncertainty in future population growth and planning of a flood protection system under uncertain climate change scenarios. Based on these idealized examples, it is shown that for flexible systems a lower initial capacity of the system is recommendable, while for inflexible systems a conservative design (with high safety factors) should be applied. The value of flexibility is shown to be high when significant learning is expected in the future, i.e. if information gathered in the future significantly reduces uncertainty.

Managing uncertainty in design flood magnitude: Flexible protection strategies versus safety factors

“…Some authorities complement the simpler rule‐based planning with the more optimal risk‐based planning by using a “zoning” approach, with different regions being assigned a different design flood (Kind, ). An initial investigation towards a risk‐based assessment is reported in Špačková and Straub (), taking basis in POMDP.…”

“…The flexibility ϕ of flood protection systems is defined in terms of their cost. Following Špačková and Straub (), it is where c 0 ( v ) is the cost of establishing the system to capacity v initially, that is, when the system is first implemented, and c i ( v i − 1 , v i ) is the undiscounted cost of adjusting the protection capacity at a future time from v i − 1 to v i . ϕ = 0 corresponds to an inflexible system and ϕ = 1 to a fully flexible system.…”

“…Illustration of the costs for fully flexible versus inflexible systems (Špačková & Straub, ): (a) initial cost c 0 when the system is first implemented. (b) Cost c i for adjustment of the system to a higher capacity v i .…”

“…Because of the Markovian nature of the uncertainty, the decision problem corresponds to the class of partially observable Markovian decision processes (POMDP). The full solution could therefore alternatively be found by application of POMDP algorithms for discrete decision problems, as suggested in Špačková and Straub (). However, the continuous domains of random variables and optimisation parameters would need to be discretised, leading to a computationally costly POMDP solution.…”

“…The theory has been used to optimise adaptation to climate change (Garrè & Friis‐Hansen, ; Hobbs, ; Simpson et al, ), yet never in the form of a comprehensive study on flood management. Our framework allows one to both quantify the value of flexibility (Špačková & Straub, )—the savings that can be incurred by a more flexible measure—as well as the value of information (Winkler, Murphy, & Katz, )—that is, the savings that can be incurred by reducing parameter uncertainty.…”