Deep Reinforcement Learning-Based Job Shop Scheduling of燬mart燤anufacturing

Abstract

Industry 4.0 production environments and smart manufacturing systems integrate both the physical and decision-making aspects of manufacturing operations into autonomous and decentralized systems. One of the key aspects of these systems is a production planning, specifically, Scheduling operations on the machines. To cope with this problem, this paper proposed a Deep Reinforcement Learning with an Actor-Critic algorithm (DRLAC). We model the Job-Shop Scheduling Problem (JSSP) as a Markov Decision Process (MDP), represent the state of a JSSP as simple Graph Isomorphism Networks (GIN) to extract nodes features during scheduling, and derive the policy of optimal scheduling which guides the included node features to the best next action of schedule. In addition, we adopt the Actor-Critic (AC) network's training algorithm-based reinforcement learning for achieving the optimal policy of the scheduling. To prove the proposed model's effectiveness, first, we will present a case study that illustrated a conflict between two job scheduling, secondly, we will apply the proposed model to a known benchmark dataset and compare the results with the traditional scheduling methods and trending approaches. The numerical results indicate that the proposed model can be adaptive with real-time production scheduling, where the average percentage deviation (APD) of our model achieved values between 0.009 and 0.21 compared with heuristic methods and values between 0.014 and 0.18 compared with other trending approaches.

Artificial Intelligence to Solve Production Scheduling Problems in Real Industrial Settings: Systematic Literature Review

“…The authors tested the solution in a real industrial case, and the algorithm turned out to provide a lower disassembly time than other algorithms, such as a GA, an improved discrete bee algorithm, and a dueling DQN. RL-Q-learning x [51] RL-AC algorithm x [52] RL-Q-learning x [53] RL-Q-learning + CTPNs x [54] RL-Q-learning x [55] MARL-Deep RL x [56] MARL-Deep RL x [57] MARL-SARSA x [58] RL-DQN x…”

“…A different type of problem and a different algorithm were presented by Elsayed, E.K, et al [51], who adopted the actor-critic (AC) network's training algorithm-based RL for achieving the optimal policy for the JSSP. The algorithm was tested in a real case, where scheduling was previously conducted following FIFO logic; the proposed algorithm achieved better results in terms of the makespan, by going from 97 UT to 60 UT.…”