A Reinforcement Learning-Variable Neighborhood Search Method for the Cloud Manufacturing Scheduling Robust Optimization Problem with Uncertain Service Time

Abstract

Cloud manufacturing (CMfg) is an advanced networked intelligent manufacturing model, which includes a large number of new product customization services. Since many products lack historical data on service time, there is uncertainty about CMfg product service time, thus, CMfg service platforms need to perform robust scheduling of CMfg services for new products. In this paper, a CMfg scheduling model considering service time uncertainty and nonpredefined service paths is constructed, and its robust equivalent is derived. In order to effectively solve the above model, this paper proposes a reinforcement learning-variable neighborhood search algorithm (rVNS) based on the variable neighborhood search algorithm, in which the upper confidence bound algorithm (UCB1) is used to adaptively select the neighborhood operator. To solve the problem of insufficient historical data at its cold start, the SARSA (lambda) method is used in this paper. In addition, this paper leverages adaptive windows to estimate and detect changes in rewards in data streams to obtain more accurate reward estimates. A large number of experiments prove that the algorithm designed in this paper has high accuracy and speed advantages in solving this problem.