Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems

Zhang, Yi; Zhu, Haihua; Tang, Dunbing; Zhou, Tong; Gui, Yong

doi:10.1016/j.rcim.2022.102412

Cited by 86 publications

(30 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(iii) It can be found that the DRL significantly outperforms all scheduling rules when trained on small-scale instances and generalized on large-scale instances, indicating that the method proposed in this study is effective when dealing with highdimensional input space; and for the whole learning process, DMU is the data used for testing, and it can be seen from the experimental data that the method proposed in this study can effectively learn to generate better for invisible instances solutions. (iv) Tested with the same parameters, the PPO algorithm [44] performs better on instances than DQN [41] and DDPG [58] and performs about the same as the metaheuristic on instances with a relatively small total number of JXMs but for larger instances, the performance of the method proposed in this study is significantly better. However, overall, regardless of the method used, the ability to solve large-scale problems is worse than the ability to solve small-scale problems, and the training error increases as the scale increases in comparison to DRL.…”

Section: Resultsmentioning

confidence: 84%

“…Liu et al [43] introduced an actor-critic deep reinforcement learning approach grounded on scheduling rules to determine actions. Zhang [44] presented an AI scheduler adaptive learning strategy rooted in the proximal policy optimization (PPO) algorithm to enhance decision-making capabilities amidst order and resource disruptions. Han [45] presented a deep reinforcement learning (DRL) framework that leverages analytical graph scheduling to navigate the complex and dynamic production environment inherent to dynamic job shop scheduling problems.…”

Section: Dynamic Job Shop Scheduling Based On Artificial Intelligence...mentioning

confidence: 99%

See 1 more Smart Citation

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Pu,

Li,

Rahimifard

2024

Sustainability

View full text Add to dashboard Cite

In response to the challenges of dynamic adaptability, real-time interactivity, and dynamic optimization posed by the application of existing deep reinforcement learning algorithms in solving complex scheduling problems, this study proposes a novel approach using graph neural networks and deep reinforcement learning to complete the task of job shop scheduling. A distributed multi-agent scheduling architecture (DMASA) is constructed to maximize global rewards, modeling the intelligent manufacturing job shop scheduling problem as a sequential decision problem represented by graphs and using a Graph Embedding–Heterogeneous Graph Neural Network (GE-HetGNN) to encode state nodes and map them to the optimal scheduling strategy, including machine matching and process selection strategies. Finally, an actor–critic architecture-based multi-agent proximal policy optimization algorithm is employed to train the network and optimize the decision-making process. Experimental results demonstrate that the proposed framework exhibits generalizability, outperforms commonly used scheduling rules and RL-based scheduling methods on benchmarks, shows better stability than single-agent scheduling architectures, and breaks through the instance-size constraint, making it suitable for large-scale problems. We verified the feasibility of our proposed method in a specific experimental environment. The experimental results demonstrate that our research can achieve formal modeling and mapping with specific physical processing workshops, which aligns more closely with real-world green scheduling issues and makes it easier for subsequent researchers to integrate algorithms with actual environments.

show abstract

Section: Resultsmentioning

confidence: 84%

Section: Dynamic Job Shop Scheduling Based On Artificial Intelligence...mentioning

confidence: 99%

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Pu,

Li,

Rahimifard

2024

Sustainability

View full text Add to dashboard Cite

show abstract

“…Cai et al [25] developed a large-scale and dynamic job-shop simulation platform. Zhang et al [26] proposed a multi-intelligence architecture system integrating self-organizing negotiation and self-learning strategies.…”

Section: Intelligent Production Scheduling Methodsmentioning

confidence: 99%

Research on an Adaptive Real-Time Scheduling Method of Dynamic Job-Shop Based on Reinforcement Learning

et al. 2022

Self Cite

View full text Add to dashboard Cite

With the rapid development of modern industrialization in our country and the continuous improvement of people’s living standards, the changing market has put forward new requirements for traditional manufacturing enterprises. The order product demand of many manufacturing enterprises is changing from a single variety, large batch to multiple varieties, small batch. In view of this change, the traditional job-shop scheduling method is far from enough, which greatly affects the efficiency of the production job-shop. In order to solve the above problems, this paper proposes a real-time scheduling method based on reinforcement learning applied in the dynamic job-shop and a new type of neural network is designed at the same time. The neural network is designed with the high-dimensional data in the above problem as input, and a policy-based reinforcement learning algorithm is proposed based on this network. In the process of research, it was found that the reinforcement learning method not only enables the agent to use historical data for learning, but also enables it to explore and learn other possible high-reward actions within a certain range, so as to realize the optimization of production goals under real-time scheduling. The effectiveness of the proposed real-time scheduling method is verified by comparing with other common rule-based scheduling methods in the manufacturing environment.

show abstract

“…Less works consider mobility with applications using AGVs or mobile robots, i.e., [111] and in some cases, they are still in a very conceptual implementation stage [90]. In [107], it is presented an approach based on multi-agent negotiation that showcases production self-organization in a smart workshop, AGVs are in charge of the transportation of material from the warehouse to production cells. Fig.…”

Section: B Automation Levelmentioning

confidence: 99%

Self-Organization in Smart Manufacturing— Background, Systematic Review, Challenges and Outlook

et al. 2023

View full text Add to dashboard Cite

The concept of smart manufacturing has attracted huge attention in the last years as an answer to the increasing complexity, heterogeneity, and dynamism of manufacturing ecosystems. This vision embraces the notion of autonomous and self-organized elements, capable of self-management and self-decision-making under a context-aware and intelligent infrastructure. While dealing with dynamic and uncertain environments, these solutions are also contributing to generating social impact and introducing sustainability into the industrial equation thanks to the development of task-specific resources that can be easily adapted, re-used, and shared. A lot of research under the context of self-organization in smart manufacturing has been produced in the last decade considering different methodologies and developed under different contexts. Most of these works are still in the conceptual or experimental stage and have been developed under different application scenarios. Thus, it is necessary to evaluate their design principles and potentiate their results. The objective of this paper is threefold. First, to introduce the main ideas behind self-organization in smart manufacturing. Then, through a systematic literature review, describe the current status in terms of technological and implementation details, mechanisms used, and some of the potential future research directions. Finally, the presentation of an outlook that summarizes the main results of this work and their interrelation to facilitate the development of self-organized manufacturing solutions. By providing a holistic overview of the field, we expect that this work can be used by academics and practitioners as a guide to generate awareness of possible requirements, industrial challenges, and opportunities that future self-organizing solutions can have towards a smart manufacturing transition.

show abstract

Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems

Cited by 86 publications

References 40 publications

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Research on an Adaptive Real-Time Scheduling Method of Dynamic Job-Shop Based on Reinforcement Learning

Self-Organization in Smart Manufacturing— Background, Systematic Review, Challenges and Outlook

Contact Info

Product

Resources

About