Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning

Song, Wen; Chen, Xinyang; Li, Qiqiang; Cao, Zhiguang

doi:10.1109/tii.2022.3189725

Cited by 90 publications

(36 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In ref. [91], Song et al. propose a DRL‐based method to automatically learn high‐quality PDRs for FJSSP end‐to‐end, without relying on existing manual PDRs.…”

Section: Job Shop Scheduling Problemsmentioning

confidence: 99%

“…In ref. [91], Song et al propose a DRL-based method to automatically learn high-quality PDRs for FJSSP end-to-end, without relying on existing manual PDRs. They design a heterogeneous graph structure to represent machine and job status during the construction process, and propose a heterogeneous graph attention network to extract graph embeddings for decision making.…”

Section: Learning To Solve Jssp With Flexible Machinesmentioning

confidence: 99%

See 1 more Smart Citation

A review on learning to solve combinatorial optimisation problems in manufacturing

Zhang

et al. 2023

IET Collab Intel Manufact

Self Cite

View full text Add to dashboard Cite

An efficient manufacturing system is key to maintaining a healthy economy today. With the rapid development of science and technology and the progress of human society, the modern manufacturing system is becoming increasingly complex, posing new challenges to both academia and industry. Ever since the beginning of industrialisation, leaps in manufacturing technology have always accompanied technological breakthroughs from other fields, for example, mechanics, physics, and computational science. Recently, machine learning (ML) technology, one of the crucial subjects of artificial intelligence, has made remarkable progress in many areas. This study thoroughly reviews how ML, specifically deep (reinforcement) learning, motivates new ideas for addressing challenging problems in manufacturing systems. We collect the literature targeting three aspects: scheduling, packing, and routing, which correspond to three pivotal cooperative production links of today's manufacturing system, that is, production, packing, and logistics respectively. For each aspect, we first present and discuss the state-of-the-art research. Then we summarise and analyse the development trends and point out future research opportunities and challenges. K E Y W O R D Sbin packing, combinatorial optimisation, deep reinforcement learning, job shop scheduling, manufacturing systems, vehicle routing | INTRODUCTIONCombinatorial optimisation problems (COPs), as one important branch of mathematical optimisation, have practical applications in many fields, such as communication, transportation, manufacturing and aroused broad research in industrial engineering, computer science, and operations research. Due to the NP (non-deterministic polynomial-time) hardness, finding their optimal solutions is challenging. In specific, the discrete solution space in COPs renders the optimisation less efficient, without the guidance of gradient as in continuous optimisation. Meanwhile, the complexity of searching the (near-)optimal solution(s) among feasible solutions could exponentially increase as the problem scale grows. Classic methods, including exact algorithms and (meta-)heuristics, generally depend on massive expertise and tuning work to solve specific problems. They are Cong Zhang, Yaoxin Wu, and Yining Ma are equal contribution.

show abstract

“…In ref. [91], Song et al. propose a DRL‐based method to automatically learn high‐quality PDRs for FJSSP end‐to‐end, without relying on existing manual PDRs.…”

Section: Job Shop Scheduling Problemsmentioning

confidence: 99%

Section: Learning To Solve Jssp With Flexible Machinesmentioning

confidence: 99%

A review on learning to solve combinatorial optimisation problems in manufacturing

Zhang

et al. 2023

IET Collab Intel Manufact

Self Cite

View full text Add to dashboard Cite

show abstract

“…Here, three typical dispatching rules, namely FIFO, LPT, and SPT are adopted and described in Table 2. [47][48][49] Thus, the action can be defined as a t = {FIFO = 1, SPT = 2; LPT = 3}. With the defined actions, the job could be chosen through specified rules and assigned to idle station at time t.…”

Section: Formulation Of Cpfjspmentioning

confidence: 99%

Digital twin and deep reinforcement learning enabled real-time scheduling for complex product flexible shop-floor

Chang

Jia

et al. 2022

Proceedings of the Institution of Mechanical Engineers, Part B:

View full text Add to dashboard Cite

Real-time scheduling methods are essential and critical to complex product flexible shop-floor due to the dynamic events in the production process, such as new job insertions, machine breakdowns and frequent rework. Recently, digital twin (DT) technology can help identify disturbances by continuously comparing physical space with virtual space, which enables real-time scheduling and greatly reduces the deviation between pre-schedule and actual schedule. However, the conventional scheduling models and algorithms cannot satisfy the adaptiveness and timeliness requirements of optimization in DT enabled shop-floor (DTS). To address above challenges, an overall framework of DT enabled real-time scheduling (DTE-RS) for complex product shop-floor is proposed to effectively reduce adverse impacts of the dynamic disturbances and minimize the makespan. Firstly, complex product flexible job shop scheduling problem (CPFJSP) is formulated as Markov Decision Process (MDP), taking into account machine breakdown and new job insertions. Then, deep Q-network (DQN) based solution is developed to achieve optimal task dispatching according to real-time production state. Finally, the case study for aircraft overhaul shop-floor is conducted to demonstrate effectiveness and feasibility of the proposed real-time scheduling method. Through experimental comparison, it is indicated that the proposed method could effectively respond to dynamic disturbances and outperform the dynamic scheduling method in terms of makespan.

show abstract

“…In recent years, researchers have explored the use of Deep Reinforcement Learning (DRL) to discover higher quality PDRs. Zhang et al [3] proposed an approach combining Graph Neural Networks (GNN) and DRL to find PDRs for Job-Shop Scheduling Problems (JSP) by modeling the workshop state with a disjunctive graph. In FJSP, in addition to considering operations, PDRs also need to consider machines selection.…”

Section: Introductionmentioning

confidence: 99%

A multiobjective flexible job shop scheduling method based on deep reinforcement learning

Liu,

Chen,

Zhang

et al. 2024

Ninth International Symposium on Sensors, Mechatronics, and Automation System (ISSMAS 2023)

View full text Add to dashboard Cite

To address the multi-objective flexible job shop scheduling problem, this paper establishes a Markov Decision Process for Multi-objective Flexible Job-Shop Scheduling Problem, aiming to minimize makespan and total machine load. Additionally, a multi-objective optimization algorithm base deep reinforcement learning is proposed. Firstly, a multi-objective decomposition strategy is introduced to decompose the problem into a set of subproblems, and graph neural networks with identical structures are employed to represent the priority dispatching rules for each subproblem. Experimental results demonstrate that this approach exhibits better generalization, higher solution quality, and faster convergence compared to previous methods.

show abstract

Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning

Cited by 90 publications

References 36 publications

A review on learning to solve combinatorial optimisation problems in manufacturing

A review on learning to solve combinatorial optimisation problems in manufacturing

Digital twin and deep reinforcement learning enabled real-time scheduling for complex product flexible shop-floor

A multiobjective flexible job shop scheduling method based on deep reinforcement learning

Contact Info

Product

Resources

About