Distributed Deep Reinforcement Learning Resource Allocation Scheme For Industry 4.0 Device-To-Device Scenarios

Abstract

This paper proposes a distributed deep reinforcement learning (DRL) methodology for autonomous mobile robots (AMRs) to manage radio resources in an indoor factory with no network infrastructure. Hence, deep neural networks (DNN) are used to optimize the decision policy of the robots, which will make decisions in a distributed manner without signalling exchange. To speed up the learning phase, a centralized training is adopted in which a single DNN is trained using the experience from all robots. Once completed, the pre-trained DNN is deployed at all robots for distributed selection of resources. The performance of this approach is evaluated and compared to 5G NR sidelink mode 2 via simulations. The results show that the proposed method achieves up to 5% higher probability of successful reception when the density of robots in the scenario is high.

Distributed Channel Allocation for Mobile 6G Subnetworks via Multi-Agent Deep Q-Learning

“…Samples for updating the DDQN weights are then drawn randomly from the buffer thereby eliminating correlations between successive samples. The agents are trained using the reward function in (11) with ζ target = 0 bps/Hz.…”

“…To overcome this limitation, algorithms for resource allocation have been traditionally based on hard-coded heuristics [5] or using optimization techniques such as game theory [6], genetic algorithm [7] and geometric programming [8]. Over the last few years, the focus appears to have shifted towards machine learning-based algorithms [4] resulting in a large number of published works applying supervised [9], unsupervised [10] and reinforcement learning techniques [11] for resource allocation in different types of wireless systems.…”

Deep Reinforcement Learning for Joint User Association and Resource Allocation in Factory Automation

“…Reinforcement learning (RL) is adopted to resolve the joint user association and channel assignment problem. The authors in [8] use RL in a factory setting for time-slot selection for packet transmission. However, in [6]- [8], the authors do not take into consideration power allocation, which is certainly beneficial to manage the interference.…”

Intelligent Multi-Agent Resource Allocation in 6G in-X Subnetworks with Limited Sensing Information

“…To overcome these limitations, we conjecture that reinforcement learning (RL) methods [10], [11] can be developed to perform resource selection, with potential performance improvement over existing approaches even with only quantized information. Moreover, a RL based method will eliminate the offline data generation requirement for the method in [9].…”