Multi-agent reinforcement learning for online scheduling in smart factories

Zhou, Tong; Tang, Dunbing; Zhu, Haihua; Zhang, Zequn

doi:10.1016/j.rcim.2021.102202

Cited by 59 publications

(23 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, one of the great advantages of multi-agent systems is the distribution of a complex task into multiple simpler tasks among different agents to avoid the exponential growth of the joint action space [92,93]. However, multi-agent RL systems present different challenges such as agent heterogeneity, communication, the definition of collective goals (cooperation), scalability, design of compact representations of the true state of the environment, and the main problem of the non-stationarity [86,87].…”

Section: Discussionmentioning

confidence: 99%

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Rodríguez

Kubler²,

Giorgio³

et al. 2022

Robotics and Computer-Integrated Manufacturing

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Rodríguez

Kubler²,

Giorgio³

et al. 2022

Robotics and Computer-Integrated Manufacturing

View full text Add to dashboard Cite

“… Zhou et al (2021) proposed a smart factory comprising varieties of components and developed a multi-agent actor–critic approach for decentralized job scheduling. The machines each correspond with an actor–critic agent (enhanced also with target policy and target critic networks), which can all observe all the states of the other agents ( via communication) in order to deal with the dynamic environment and also deal with non-stationarity.…”

Section: Applicationsmentioning

confidence: 99%

“…However, the corresponding information sharing costs reduce the scalability and efficiency of this approach. A close look at the most comprehensive approaches that incorporate a larger number of agents, such as Kim et al (2020) and Zhou et al (2021) , demonstrates the fact that scalable approaches tend to use more sparse communication strategies where agents are selectively chosen to exchange information to save time and cost. A more advanced approach, however, is the use of negotiation-based strategies incorporated in Wang (2020) and Kim et al (2020) , which can even be enhanced by taking advantage of negotiation learning.…”

Section: Applicationsmentioning

confidence: 99%

A review of the applications of multi-agent reinforcement learning in smart factories

Bahrpeyma

Reichelt

2022

Front. Robot. AI

View full text Add to dashboard Cite

The smart factory is at the heart of Industry 4.0 and is the new paradigm for establishing advanced manufacturing systems and realizing modern manufacturing objectives such as mass customization, automation, efficiency, and self-organization all at once. Such manufacturing systems, however, are characterized by dynamic and complex environments where a large number of decisions should be made for smart components such as production machines and the material handling system in a real-time and optimal manner. AI offers key intelligent control approaches in order to realize efficiency, agility, and automation all at once. One of the most challenging problems faced in this regard is uncertainty, meaning that due to the dynamic nature of the smart manufacturing environments, sudden seen or unseen events occur that should be handled in real-time. Due to the complexity and high-dimensionality of smart factories, it is not possible to predict all the possible events or prepare appropriate scenarios to respond. Reinforcement learning is an AI technique that provides the intelligent control processes needed to deal with such uncertainties. Due to the distributed nature of smart factories and the presence of multiple decision-making components, multi-agent reinforcement learning (MARL) should be incorporated instead of single-agent reinforcement learning (SARL), which, due to the complexities involved in the development process, has attracted less attention. In this research, we will review the literature on the applications of MARL to tasks within a smart factory and then demonstrate a mapping connecting smart factory attributes to the equivalent MARL features, based on which we suggest MARL to be one of the most effective approaches for implementing the control mechanism for smart factories.

show abstract

“…Message scheduling is a process in which action is carried out by a distributed system's broker (scheduler) mechanism [13], it makes use of message contexts or any other type of information that may be considered according to their priorities. Messages are categorized in the proposed system based on their features and quality of service (QoS) specifications, and applications are divided into two categories: critical and non-critical.…”

Section: Introductionmentioning

confidence: 99%

Message Scheduling in Blockchain Based IoT Environment With Additional Fog Broker Layer

et al. 2022

View full text Add to dashboard Cite

Recently researchers and companies have shown significant interest in merging blockchain and the Internet of Things (IoT) to create a safe, reliable, and resilient communication platform. However, determining the proper role of blockchain in existing IoT contexts with minimum implications is a challenge. This work suggests a message schedule for a blockchain-based architecture with two access-level setting filters for incoming messages: critical and non-critical. The proposed work of the researchers divides the fog layer into two parts: action clusters and blockchain fog clusters. Similar to the three-layered IoT architecture, the action cluster and the main cloud data center work together for critical message requests. The blockchain fog cluster is dedicated to only the blockchain application's requirements. In the fog layer, a fog broker is used to schedule critical and non-critical messages in the action and blockchain fog clusters, respectively. The proposed technique is compared to the existing Dual Fog-IoT architecture. The solution is also tested for fog and cloud computing resource utilization. The findings demonstrate that this architecture is feasible for varying percentages of receiving critical and non-critical messages. In addition to the inherent benefits of blockchain, the suggested paradigm reduces the system loss rate and offloads the cloud data center with minimal changes to the existing IoT ecosystem.INDEX TERMS Internet of Things (IoT), Message Scheduling, Wireless Sensor Networks, Fog Computing.

show abstract

Multi-agent reinforcement learning for online scheduling in smart factories

Cited by 59 publications

References 29 publications

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

A review of the applications of multi-agent reinforcement learning in smart factories

Message Scheduling in Blockchain Based IoT Environment With Additional Fog Broker Layer

Contact Info

Product

Resources

About