Planning Under Uncertainty Applications in Power Plants Using Factored Markov Decision Processes

Abstract

Due to its ability to deal with non-determinism and partial observability, represent goals as an immediate reward function and find optimal solutions, planning under uncertainty using factored Markov Decision Processes (FMDPs) has increased its importance and usage in power plants and power systems. In this paper, three different applications using this approach are described: (i) optimal dam management in hydroelectric power plants, (ii) inspection and surveillance in electric substations, and (iii) optimization of steam generation in a combined cycle power plant. For each case, the technique has demonstrated to find optimal action policies in uncertain settings, present good response and compilation times, deal with stochastic variables and be a good alternative to traditional control systems. The main contributions of this work are as follows, a methodology to approximate a decision model using machine learning techniques, and examples of how to specify and solve problems in the electric power domain in terms of a FMDP.

An MDP Model for Integrated Operational Decision Making Under Uncertainty in a Gas and Oil Separation Plant

“…MDP is a stochastic dynamic programming technique, which is used for decision making under uncertainty [24] with very wide range of real-life applications and efficient solution techniques [25] [24]. The main advantages of MDP include: 1) its ability to obtain optimal policies over finite and infinite planning horizons, where the latter is used to model stationary systems, 2) MDP can have different cost criteria that can depend on the initial system state, the next system state or both [24], 3) MDP is used in real-life for decision making under uncertainty [25] [26]. The main disadvantages of MDP are: 1) the difficulty to obtain their state transition probabilities [27], [28], because they require large number of statistical data [29], 2) establishing the cost/reward criteria and 3) the lack of standard software packages to to explicitly solve for individual modeling problems (i.e.…”

An Optimal Phase Arrangement of Distribution Transformers under Risk Assessment

“…Hence, PSO tries to reach compatibility between local search and global search. In the FPSO system, the position and velocity of particles is defined in Equation (19) and Equation (20) [26].…”

“…In this paper, the Monte Carlo method [19,20] is used to calculate the generation value-at-risk (VAR) of presented WTs/PVs. Using the historical data of wind speed and global radiation, the generation VAR of WTs/PVs is derived under a given level of confidence.…”