Nested Markov chain — A novel approach to model network-induced constraints

Jia, Pengyi; Wang, Xianbin

doi:10.1109/iemcon.2017.8117160

Cited by 2 publications

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To tackle with this challenge, we first design a joint sensing and transmission protocol to enable the joint sensing and transmission for UAVs in different modes. This protocol can be analyzed by nested bi-level Markov chains [11], in which sensing and transmission processes are formulated as the state transitions in the Markov chains. Therefore, the UAV trajectory design problem can be regarded as a Markov decision problem (MDP) [12], [13].…”

mentioning

confidence: 99%

Cellular UAV-to-Device Communications: Trajectory Design and Mode Selection by Multi-Agent Deep Reinforcement Learning

Zhang

et al. 2020

IEEE Trans. Commun.

View full text Add to dashboard Cite

In the current unmanned aircraft systems (UASs) for sensing services, unmanned aerial vehicles (UAVs) transmit their sensory data to terrestrial mobile devices over the unlicensed spectrum.However, the interference from surrounding terminals is uncontrollable due to the opportunistic channel access. In this paper, we consider a cellular Internet of UAVs to guarantee the Quality-of-Service (QoS), where the sensory data can be transmitted to the mobile devices either by UAV-to-Device (U2D) communications over cellular networks, or directly through the base station (BS). Since UAVs' sensing and transmission may influence their trajectories, we study the trajectory design problem for UAVs in consideration of their sensing and transmission. This is a Markov decision problem (MDP) with a large state-action space, and thus, we utilize multi-agent deep reinforcement learning (DRL) to approximate the state-action space, and then propose a multi-UAV trajectory design algorithm to solve this problem.Simulation results show that our proposed algorithm can achieve a higher total utility than policy gradient algorithm and single-agent algorithm. ). I. INTRODUCTIONWith high mobility and low operational cost, unmanned aerial vehicle (UAV) is recognized as a powerful facility to provide sensing services [2], [3], which has found use in a wide range of sensing applications including traffic monitoring [4], industrial inspection [5], precision agriculture [6], and fire surveillance [7]. In the current unmanned aircraft systems (UASs) for sensing services, UAVs transmit their sensory data to terrestrial mobile devices 1 over the unlicensed spectrum [8]. However, due to the opportunistic channel access at the media access control (MAC) layer, the interference from surrounding terminals is uncontrollable, and the Quality-of-Service (QoS) for sensing services cannot be guaranteed. Therefore, a more reliable network is necessary.As an effective solution, terrestrial cellular networks can be utilized to provide UAV sensing services with guaranteed QoS, which we refer to as the cellular Internet of UAVs [9], [10]. In the cellular Internet of UAVs, UAVs first transmit the sensory data to the BS, after which the BS sends the received data to mobile devices. When UAVs fly far from their sensing targets, they may suffer low sensing quality, while if UAVs are far from the BS, it is difficult to transmit their sensory data back to the BS. Therefore, to ensure the QoS, UAVs should jointly take sensing and transmission into consideration in the designing of their trajectories. However, when UAVs are close to their mobile devices, the data rate can be further improved if UAVs can directly transmit the sensory data to proximal mobile devices, namely the UAV-to-Device (U2D) communications, rather than transmit through cellular communications.In this paper, we propose to enable U2D communications in the cellular Internet of UAVs.We consider a orthogonal frequency division multiple access (OFDMA) cellular Internet of UAVs 2 , where the sensory data ca...

show abstract

mentioning

confidence: 99%