Deep Reinforcement Learning Multi-Agent System for Resource Allocation in Industrial Internet of Things

Abstract

The high number of devices with limited computational resources as well as limited communication resources are two characteristics of the Industrial Internet of Things (IIoT). With Industry 4.0 emerges a strong demand for data processing in the edge, constrained primarily by the limited available resources. In industry, deep reinforcement learning (DRL) is increasingly used in robotics, job shop scheduling and supply chain. In this work, DRL is applied for intelligent resource allocation for industrial edge devices. An optimal usage of available resources of the IIoT devices should be achieved. Due to the structure of IIoT systems as well as security aspects, multi-agent systems (MASs) are preferred for decentralized decision-making. In our study, we build a network from physical and virtualized representative IIoT devices. The proposed approach is capable of dealing with several dynamic changes of the target system. Three aspects are considered when evaluating the performance of the MASs: overhead due to the MASs, improvement of the resource usage of the devices as well as latency and error rate. In summary, the agents’ resource usage with respect to traffic, computing resources and time is very low. It was confirmed that the agents not only achieve the desired results in training but also that the learned behavior is transferable to a real system.

Virtual Commissioning of Distributed Systems in the Industrial Internet of Things

“…In addition to the decision about the deployment on virtual or physical devices, another relevant use case is the optimization of task distribution on networked computing units with given requirements, such as hard real-time, permissions, priority, and available resources. The usage of simulation in this context is explained by means of an agent-based resource allocation for logic-layer algorithms in DSPSs, as proposed in [ 83 ]. Agents decide about the available resources or routing to neighbors.…”

“…The IIoT is a highly complex network, with the occurrence of dynamic changes as one of its main characteristics. Three kinds of changes are presented in [ 83 ]: changes in the network topology, changes in the streaming data, and changes in the data analysis tasks (see Section 2.1 ). Thus, adaptivity to dynamic changes is a relevant requirement for algorithms executed in the IIoT.…”

“…For evaluation, the exemplary use case of an RL-based MAS for intelligent resource allocation in IIoT with edge computing is examined. It comprises both the MAS as enabler for DSPSs [ 83 ] on the middleware layer as well as different data stream processing algorithms [ 4 , 22 , 88 ] that belong to the logic layer. While the single approaches have already been evaluated individually, the proposed application of simulation is used for a realistic evaluation of the overall system and the interaction of the single approaches in combination with an industrial plant.…”

Optimal Task Allocation Algorithm Based on Queueing Theory for Future Internet Application in Mobile Edge Computing Platform

“…Therefore, various uses of MEC have been proposed [52][53][54][55]. Especially, machine learning and artificial intelligence are effective in a MEC platform [56][57][58][59]. For utilizing machine learning and artificial intelligence, a large number of data sets obtained from the real environment and a long training time to determine an appropriate task allocation.…”

Edge-Computing-Based Intelligent IoT: Architectures, Algorithms and Applications

“…For the first category-task and resource management for edge computing-Yang et al [1] investigated the problem of task offloading for mobile edge-computing networks; they proposed a deep-supervised-learning-based computational offloading (DSLO) algorithm to jointly optimise the problems of offloading decisions and bandwidth allocation. Rosenberger et al [2] studied the problem of resource allocation in the industrial Internet of Things (IIoT); they proposed a multi-agent deep-reinforcement-learning (MARL)-based strategy which can deal with several dynamic changes in the target system and achieve the optimal usage of available resources for IIoT devices.…”