Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning

Abstract

We propose a framework to learn to schedule a job-shop problem (JSSP) using a graph neural network (GNN) and reinforcement learning (RL). We formulate the scheduling process of JSSP as a sequential decision-making problem with graph representation of the state to consider the structure of JSSP. In solving the formulated problem, the proposed framework employs a GNN to learn that node features that embed the spatial structure of the JSSP represented as a graph (representation learning) and derive the optimum scheduling policy that maps the embedded node features to the best scheduling action (policy learning). We employ Proximal Policy Optimization (PPO) based RL strategy to train these two modules in an end-to-end fashion. We empirically demonstrate that the GNN scheduler, due to its superb generalization capability, outperforms practically favored dispatching rules and RL-based schedulers on various benchmark JSSP. We also confirmed that the proposed framework learns a transferable scheduling policy that can be employed to schedule a completely new JSSP (in terms of size and parameters) without further training.

Deep reinforcement learning in production systems: a systematic literature review

“…It was able to outperform conventional heuristics such as shortest setup or processing time and reduced total makespan and computation times after training. A further increase in generalisation was reached by Park et al (2021) by applying a graph neural network (GNN). The GNN learned the basic spatial structure of the problem in form of a graph that could be transferred to new problems and adapted its mapped policy.…”

ScheduleNet: Learn to solve multi-agent scheduling problems with reinforcement learning

“…For example, [9,23] have proposed to learn scheduling policy for each agent and hence, it requires an additional training to solve JSPs with a different number of agents from the training cases. Recently, [30,45] have proposed to learn a shared scheduling policy for all agents while utilizing the disjunctive graph representation of JSP. Unlike these methods utilizing well-designed dense reward, we directly use the makespan reward to train a policy.…”

“…We evaluate the scheduling performance to verify ScheduleNet's generalization capacity to unseen JSP distributions on the Taillard's 80 dataset [39]. We compare against two deep RL baselines [30,45], as well as the mentioned heuristics (MOR, FIFO, and SPT).…”

“…Job-shop scheduling makespans on TA dataset (Part 1)Instance N × m SPT FIFO MOR Park et al[30] Zhang et al[45] ScheduleNet OPT…”

A Global DAG Task Scheduler Using Deep Reinforcement Learning and Graph Convolution Network

“…There exist several works on applying DRL for scheduling tasks in the various domains such as cloud computing, networks, and manufacturing systems [8]- [11], [15]- [18]. DeepRM [8] was the first attempt to learn a scheduling policy systematically through DRL.…”