A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

Zhong, Chen; Lu, Ziyang; Gursoy, M. Cenk; Velipasalar, Senem

doi:10.48550/arxiv.1908.08401

Cited by 3 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently DRL has achieved significant breakthroughs in the dynamic spectrum allocation problems [27]- [35]. The works in [27], [28], [29], [30] and [31] studied the multichannel access problem under the assumption of Markov spectrum occupancy model. The authors of [27] considered the highly correlation between channels thus the user can access the vacant channel by historical partial observations.…”

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

“…The authors of [27] considered the highly correlation between channels thus the user can access the vacant channel by historical partial observations. A actorcritic DRL based framework was proposed in [28], [29] and its performance was further improved in [27] especially in scenarios with a large number of channels. In [30] all channels are independent so the user is supposed to have fully observation of the system via wideband spectrum sensing techniques.…”

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

“…The independent channel model is also adopted in [31], but the authors of [31] considered the presence of spectrum sensing errors, and the position of each user is specified in the proposed scenario. Moreover, multi-user scenarios are also studied in [29] and [31] through distributive learning. However, in the most of aforementioned works the user only selects one channel to access with the hope of avoiding collisions at each time slot.…”

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

“…However, in the most of aforementioned works the user only selects one channel to access with the hope of avoiding collisions at each time slot. The authors of [29] considered a scenario where the user can access more than one channel at a time, but it has nothing to do with the user's requirement. In other words, previous works mainly focused on allocating the single channel to the user without taking into account the user's demand of bandwidth.…”

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks

Zhang

Wang

et al. 2020

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

In this paper, the problem of dynamic spectrum sensing and aggregation is investigated in a wireless network containing N correlated channels, where these channels are occupied or vacant following an unknown joint 2-state Markov model. At each time slot, a single cognitive user with certain bandwidth requirement either stays idle or selects a segment comprising C (C < N) contiguous channels to sense. Then, the vacant channels in the selected segment will be aggregated for satisfying the user requirement. The user receives a binary feedback signal indicating whether the transmission is successful or not (i.e., ACK signal) after each transmission, and makes next decision based on the sensing channel states. Here, we aim to find a policy that can maximize the number of successful transmissions without interrupting the primary users (PUs). The problem can be considered as a partially observable Markov decision process (POMDP) due to without full observation of system environment. We implement a Deep Q-Network (DQN) to address the challenge of unknown system dynamics and computational expenses. The performance of DQN, Q-Learning, and the Improvident Policy with known system dynamics is evaluated through simulations. The simulation results show that DQN can achieve near-optimal performance among different system scenarios only based on partial observations and ACK signals.

show abstract

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

Section: B Deep Reinforcement Learning and Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks

Zhang

Wang

et al. 2020

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…Similar works using Qlearning and DQN can be found respectively in [12] and [13]. Finally, an Actor-Critic framework using deep neural network (DNN) is analyzed in [14].…”

Section: Introductionmentioning

confidence: 97%

Multi-Agent Deep Stochastic Policy Gradient for Event Based Dynamic Spectrum Access

Kassab¹,

Destounis²,

Tsilimantos³

et al. 2020

Preprint

View full text Add to dashboard Cite

We consider the dynamic spectrum access (DSA) problem where K Internet of Things (IoT) devices compete for T time slots constituting a frame. Devices collectively monitor M events where each event could be monitored by multiple IoT devices. Each device, when at least one of its monitored events is active, picks an event and a time slot to transmit the corresponding active event information. In the case where multiple devices select the same time slot, a collision occurs and all transmitted packets are discarded. In order to capture the fact that devices observing the same event may transmit redundant information, we consider the maximization of the average sum event rate of the system instead of the classical frame throughput. We propose a multi-agent reinforcement learning approach based on a stochastic version of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to access the frame by exploiting device-level correlation and time correlation of events. Through numerical simulations, we show that the proposed approach is able to efficiently exploit the aforementioned correlations and outperforms benchmark solutions such as standard multiple access protocols and the widely used Independent Deep Q-Network (IDQN) algorithm.

show abstract

Space Diversity-Based Grant-Free Random Access for Critical and Non-Critical IoT Services

Kassab

Simeone

Munari

et al. 2020

ICC 2020 - 2020 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

Communication services with heterogeneous performance requirements are emerging as key use cases for 5G and beyond. This paper deals with the coexistence of two service classes, i.e., critical service (CS) and non-critical service (NCS) on a grant-free channel consisting of the radio access and backhaul segments. On the radio access segment, Internet-of-Things (IoT) devices send packets to a set of non-cooperative access points (APs) using slotted ALOHA (SA). The APs then forward correctly received messages to a base station over a shared wireless backhaul segment adopting SA. The APs

show abstract

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

Cited by 3 publications

References 30 publications

Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks

Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks

Multi-Agent Deep Stochastic Policy Gradient for Event Based Dynamic Spectrum Access

Space Diversity-Based Grant-Free Random Access for Critical and Non-Critical IoT Services

Contact Info

Product

Resources

About