Multi-agent reinforcement learning for intelligent resource allocation in IIoT networks

The high number of devices with limited computational resources as well as limited communication resources are two characteristics of the Industrial Internet of Things (IIoT). With Industry 4.0 emerges a strong demand for data processing in the edge, constrained primarily by the limited available resources. In industry, deep reinforcement learning (DRL) is increasingly used in robotics, job shop scheduling and supply chain. In this work, DRL is applied for intelligent resource allocation for industrial edge devices. An optimal usage of available resources of the IIoT devices should be achieved. Due to the structure of IIoT systems as well as security aspects, multi-agent systems (MASs) are preferred for decentralized decision-making. In our study, we build a network from physical and virtualized representative IIoT devices. The proposed approach is capable of dealing with several dynamic changes of the target system. Three aspects are considered when evaluating the performance of the MASs: overhead due to the MASs, improvement of the resource usage of the devices as well as latency and error rate. In summary, the agents’ resource usage with respect to traffic, computing resources and time is very low. It was confirmed that the agents not only achieve the desired results in training but also that the learned behavior is transferable to a real system.

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning Multi-Agent System for Resource Allocation in Industrial Internet of Things

Rosenberger

Urlaub

Rauterberg

et al. 2022

“…In addition, many end-users still object to transferring critical enterprise and production data to remote service providers. Thus, the shift to edge computing has been recognizable for a few years in industry [ 2 , 3 , 4 , 5 ]. In the context of distributed stream processing systems (DSPSs), this trend is also evident, as the use of edge device resources for (pre)processing tasks is increasing [ 6 , 7 ].…”

Section: Introductionmentioning

confidence: 99%

Virtual Commissioning of Distributed Systems in the Industrial Internet of Things

Rosenberger

Selig

Ristić

et al. 2023

Self Cite

With the convergence of information technology (IT) and operational technology (OT) in Industry 4.0, edge computing is increasingly relevant in the context of the Industrial Internet of Things (IIoT). While the use of simulation is already the state of the art in almost every engineering discipline, e.g., dynamic systems, plant engineering, and logistics, it is less common for edge computing. This work discusses different use cases concerning edge computing in IIoT that can profit from the use of OT simulation methods. In addition to enabling machine learning, the focus of this work is on the virtual commissioning of data stream processing systems. To evaluate the proposed approach, an exemplary application of the middleware layer, i.e., a multi-agent reinforcement learning system for intelligent edge resource allocation, is combined with a physical simulation model of an industrial plant. It confirms the feasibility of the proposed use of simulation for virtual commissioning of an industrial edge computing system using Hardware-in-the-Loop. In summary, edge computing in IIoT is highlighted as a new application area for existing simulation methods from the OT perspective. The benefits in IIoT are exemplified by various use cases for the logic or middleware layer using physical simulation of the target environment. The relevance for real-life IIoT systems is confirmed by an experimental evaluation, and limitations are pointed out.

“…The densification of the network [5] is one of the key features of the 5G wireless network architecture, which not only contributes to increasing the system capacity of 5G networks, but also is closely related to user experience enhancement. As an important technique for improving the efficiency and quality of communications, dense RL method that can outperform the single agent in resource allocation, especially in the multi-cell multi-user system [34,35]. In [36], a joint resource allocation problem is settled by a MADRL relying on the independent Q-learning method [37].…”

Section: Introductionmentioning

confidence: 99%

Joint Optimization of Bandwidth and Power Allocation in Uplink Systems with Deep Reinforcement Learning

Zhang,

Lv,

Huang

et al. 2023

Wireless resource utilizations are the focus of future communication, which are used constantly to alleviate the communication quality problem caused by the explosive interference with increasing users, especially the inter-cell interference in the multi-cell multi-user systems. To tackle this interference and improve the resource utilization rate, we proposed a joint-priority-based reinforcement learning (JPRL) approach to jointly optimize the bandwidth and transmit power allocation. This method aims to maximize the average throughput of the system while suppressing the co-channel interference and guaranteeing the quality of service (QoS) constraint. Specifically, we de-coupled the joint problem into two sub-problems, i.e., the bandwidth assignment and power allocation sub-problems. The multi-agent double deep Q network (MADDQN) was developed to solve the bandwidth allocation sub-problem for each user and the prioritized multi-agent deep deterministic policy gradient (P-MADDPG) algorithm by deploying a prioritized replay buffer that is designed to handle the transmit power allocation sub-problem. Numerical results show that the proposed JPRL method could accelerate model training and outperform the alternative methods in terms of throughput. For example, the average throughput was approximately 10.4–15.5% better than the homogeneous-learning-based benchmarks, and about 17.3% higher than the genetic algorithm.