A Fast Algorithm for Large-Scale MDP-Based Systems in Smart Grid

Abstract

In this study, we investigate the fast algorithms for the Large-Scale Markov Decision Process (LSMDP) problem in smart gird. Markov decision process is one of the efficient mathematical tools to solve the control and optimization problems in wireless smart grid systems. However, the complexity and the memory requirements exponentially increase when the number of system state grows in. Moreover, the limited computational ability and small size of memory on board constraint the application of wireless smart grid systems. As a result, it is impractical to implement those LSMDP-based approaches in such systems. Therefore, we propose the fast algorithm with low computational overhead and good performance in this study. We first derive the factored MDP representation, which substitutes LSMDP in a compact way. Based on the factored MDP, we propose the fast algorithm, which considerably reduces the size of state space and remains reasonable performance compared to the optimal solution.