Two‐time scale reinforcement learning and applications to production planning

Abstract

This study is concerned with reinforcement learning enhanced by two-time scale approximations. Many systems arising in applications are large and complex. To treat these problems, it is often beneficial, and sometimes necessary, to reduce the dimensionality and aggregate states that are 'close' to each other. In this study, the authors propose a two-time scale reinforcement learning method for such an aggregation process. In particular, they present how to classify states that are 'close' and demonstrate the effectiveness of the authors' state aggregation based two-time scale methods. Thus the problem can be considered as using learning for identifying the system. A production planning problem with failure-prone machines is used throughout this study to illustrate the main ideas, key steps and results. Monte Carlo simulations are used to generate the random environment.