A Deep Reinforcement Learning Based Solution for Flexible Job Shop Scheduling Problem

Abstract

Flexible job shop Scheduling problem (FJSP) is a classic problem in combinatorial optimization and a very common form of organization in a real production environment. Traditional approaches for FJSP are ill-suited to deal with complex and changeable production environments. Based on 3D disjunctive graph dispatching, this work proposes an end-to-end deep reinforcement learning (DRL) framework. In this framework, a modified pointer network, which consists of an encoder and a decoder, is adopted to encode the operations to be scheduled according to the selected scheduling features. Then with the attention mechanism, an input is pointed as an action in each decoding step, and a recurrent neural network (RNN) is used to model the decoder network. To train the network to minimize the makespan, a policy gradient algorithm is applied to optimize its parameters. The trained model generates the scheduling solution as a sequence of consecutive actions in real-time without retraining for every new problem instance. Experimental results show that this method can obtain better performance than the classic heuristic rules when only one model is trained on all the test instances.

Gated‐Attention Model with Reinforcement Learning for Solving Dynamic Job Shop Scheduling Problem

“…where A t is the advantage function computed as the difference between the discounted sum of rewards and the baseline estimate at the state as shown in (21), r t (θ) is the ratio of the new policy to the old policy as described in (22), and is the clipping ratio.…”

“…Recently the use of DRL methods for solving scheduling problems is gaining attention and, promising results are being obtained in job shop scheduling problem (JSSP) [21], flexible job shop scheduling problem (FJSP) [22], and open shop scheduling problem (OSSP) [23]. In Ref.…”

You Only Train Once: A highly generalizable reinforcement learning method for dynamic job shop scheduling problem

“…In today's complex and varied production processes, dynamic events such as machine breakdown or the change of the processing time and machine order of jobs are inevitable to be considered, necessitating the remarkable results on various combinatorial optimization problems such as traveling salesman problem (TSP) [12], the vehicle routing problem (VRP) [13] and JSP [14]. Existing research has shown that using reinforcement learning to solve DJSP has at least four advantages: 1) RL doesn't require the complete mathematical model or the large labeled datasets of the scheduling environment, but can learn from the interaction with the environment and store the learned knowledge to achieve "offine learning and online application" [15]. 2) Unlike the exiting approaches that have to reschedule the jobs when faced dynamic events, RL can adjust the learning strategy automatically and achieve adaptive scheduling [16].…”

A deep reinforcement learning based approach for dynamic distributed blocking flowshop scheduling with job insertions

“…Machines with intelligent agents evaluate the priorities of jobs and distribute them through negotiation. Han et al [27] proposed an end-to-end DRL framework based on 3D disjunctive graph dispatching. They improved the pointer network and trained the policy with 20 static features and 24 dynamic features that described the full picture of scheduling problem.…”