Intelligent Resource Management Using Multiagent Double Deep Q-Networks to Guarantee Strict Reliability and Low Latency in IoT Network

Abstract

With the rapid adoption of the Internet of Things, it is necessary to go beyond fifth-generation applications and apply stringent high reliability and low latency requirements, closely related to strict delay demands. These requirements support massive network connectivity for multiple Internet of Things devices. Hence, in this paper, we optimize energy efficiency and achieve quality-of-service requirements by mitigating co-channel interference, performing efficient power control of transmitters, and harvesting energy using timeslot exchanges. Due to a nonconvex optimization problem, we propose an iterative algorithm for power allocation and time slot interchange to reduce the computational complexity. To achieve a high degree of ultra-reliability and low latency with quality-of-service-aware instantaneous reward under massive connectivity, we efficiently employ multiagent reinforcement learning by addressing the intelligent resource management problem via a novel Double Deep Q Network. The network prioritizes experience replay to exploit the best policy and maximize accumulative rewards. It also learns the optimal policy and enhances learning efficiency by maximizing its reward function to make decisions with high intelligence and guarantee strict ultra-reliability and low latency. The simulation result shows that the Double Deep Q Network with prioritized experience replay can guarantee stringent ultra-reliability and low latency. As a result, the cochannel interference between transmission links and the high-power consumption density associated with the massive connectivity of the Internet of Things devices are mitigated.INDEX TERMS Internet of things, beyond fifth-generation, energy efficiency, massive connectivity.

One-Bit Massive MIMO Precoding Using Unsupervised Deep Learning

“…Deep learning (DL) is a powerful technique that has shown great potential in solving optimization problems in wireless communications. Deep learning techniques can be used for a variety of tasks in wireless communications, such as active sensing, channel modeling, distributed source coding, signal detection, channel estimation, compression sensing, encoding and decoding, security and privacy, Internet of Thing (IoT) resource management with massive number of nodes and channel estimation in massive MIMO systems [21], [22], [23], [24], [25], [26]. This paper attempts to address the issue of one-bit precoding in massive MIMO systems using DL methods.…”

Empowering Energy-Sustainable IoT Devices With Harvest Energy-Optimized Deep Neural Networks

“…It was assumed that the IoT device was equipped with a PS policy to exploit the SWIPT function. The channel gain was constant during one coherence interval and varied independently in subsequent intervals [30][31][32]. The received RF signal ƴ 𝑘 ∈ ∁ of RX k , can be written as:…”

“…𝑃 𝑡𝑜𝑡𝑎𝑙 is convex with respect to 𝒫 𝑘 and ℛ 𝑘 is concave, as shown by equations (7) and (8). The efficient conventional successive convex function in (6) was obtained using the second-order derivatives of 𝒫 𝑘 and 𝜉 𝑘 [30], [34]:…”

“…The gradients of each node in (26) determined if each node's updated weights had changed. Every time the gradient of each node was calculated, the estimated gradient (30) was evaluated by updating a new randomly selected weight from (31). Then, (32) was used to obtain optimal 𝒫 𝑘 * and IoT PS ratios to RF EH 𝜉 𝑘 * in place of the true gradient.…”

“…The proposed GD for FNN was used to accelerate the learning process and estimate the gradient in each iteration [25][26][27] to prevent training failure and circumvent the zero gradient ₺(Ϣ 𝑜𝑙𝑑 , Ъ 𝑜𝑙𝑑 ) = 0 problems (30). The loss function gradient was represented using the chain rule of partial derivatives as the product of the gradients of activation functions ⨍ ᴎ,𝑙 to ƻ ᴎ,𝑙 in the hidden layers in (26) for Ϣ and Ъ.…”