Partial Computation Offloading in NOMA-Assisted Mobile-Edge Computing Systems Using Deep Reinforcement Learning

Tuong, Van Dat; Truong, Thanh Phung; Nguyen, The-Vi; Noh, Wonjong; Cho, Sungrae

doi:10.1109/jiot.2021.3064995

“…To solve the problems of spec-trum reuse and communication resource allocation in a D2Denabled MEC system, policy-based DRL optimization of power control and channel selection was efficiently executed to maximize the system capacity and spectrum efficiency in [108]. Furthermore, hybrid channel access with nonorthogonal multiple access (NOMA) and orthogonal multiple access (OMA) users was considered in [128] to achieve the optimal policy of partial offloading decisions and channel resource allocation using actor-critic DQN. In the time-varying MEC system with multiple users accessing an individual MEC server, to minimize the total energy consumption, the offloading strategy was proposed in [129] by considering the heterogeneous resource requirements and delay constraints in communication and computation.…”

Section: ) Dynamic Spectrum Accessmentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei

¹

,

Guo

²

,

Li

³

et al. 2022

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep reinforcement learning (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. INDEX TERMS Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL).

show abstract

“…Fang et al [17] proposed a low complexity algorithm to minimize the total energy consumption by the task assignment, power allocation and user association for the multi-user NOMA-MEC network. With the rapid development of artificial intelligence, many studies have been on spent for exploiting the potentials of NOMA and MEC by designing disparate learning schemes [18]- [20]. Tuong et al [18] utilized the deep Q-network and actor-critic network to reduce effectively the computational overhead by jointly optimizing the computation offloading policy and channel resource allocation in a NOMA-MEC network.…”

Section: Related Workmentioning

confidence: 99%

“…With the rapid development of artificial intelligence, many studies have been on spent for exploiting the potentials of NOMA and MEC by designing disparate learning schemes [18]- [20]. Tuong et al [18] utilized the deep Q-network and actor-critic network to reduce effectively the computational overhead by jointly optimizing the computation offloading policy and channel resource allocation in a NOMA-MEC network. Qian et al [19] proposed a cross-entropy algorithm based on the probabilistic learning, to find the optimal pairing of vehicular computering-users in NOMA aided vehicular edge computing networks.…”

Section: Related Workmentioning

confidence: 99%

Joint Multi-domain Resource Allocation and Trajectory Optimization in UAV-assisted M-IoT Networks

Qian¹

2022

Preprint

0

View full text Add to dashboard Cite

<div>The integration of Maritime Internet of Things (M-IoT) technology and unmanned aerial/surface vehicles (UAVs/USVs) has been emerging as a promising navigational information technique in intelligent ocean systems. With the unprecedented increase of computation-intensive yet latency sensitive marine mobile Internet services, mobile edge computing (MEC) and non-orthogonal multiple access (NOMA) have been envisioned as promising approaches to providing with the low-latency as well as reliable computing services and ultra-dense connectivity. In this paper, we investigate the energy consumption minimization based energy-efficient MEC via cooperative NOMA for the UAV-assisted M-IoT networks. We consider that USVs offload their computation-workload to the UAV equipped with the edge-computing server subject to the UAV mobility. To improve the energy efficiency of offloading transmission and workload computation, we focus on minimizing the total energy consumption by jointly optimizing the USVs’ offloaded workload, transmit power, computation resource allocation as well as the UAV trajectory subject to the USVs’ latency requirements. Despite the nature of mixed discrete and non-convex programming of the formulated problem, we exploit the vertical decomposition and propose a two-layered algorithm for solving it efficiently. Specifically, the top-layered algorithm is proposed to solve the problem of optimizing the UAV trajectory based on the idea of Deep Reinforcement Learning (DRL), and the underlying algorithm is proposed to optimize the underlying multi-domain resource allocation problem based on the idea of the Lagrangian multiplier method. Numerical results are provided to validate the effectiveness of our proposed algorithms as well as the performance advantage of NOMA-enabled computation offloading in terms of overall energy consumption.</div>

show abstract

“…To solve the problems of spectrum reuse and communication resource allocation in a D2D-enabled MEC system, policybased DRL optimization of power control and channel selection was efficiently executed to maximize the system capacity and spectrum efficiency in [103]. Furthermore, hybrid channel access with nonorthogonal multiple access (NOMA) and orthogonal multiple access (OMA) users was considered in [124] to achieve the optimal policy of partial offloading decisions and channel resource allocation using actor-critic DQN. In the time-varying MEC system with multiple users accessing an individual MEC server, to minimize the total energy consumption, the offloading strategy was proposed in [125] by considering the heterogeneous resource requirements and delay constraints in communication and computation.…”

Section: B Studies On Dynamic Channelmentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei,

Guo,

Li

et al. 2022

Preprint

0

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep RL (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. Index Terms-Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL). emote Areas irplane taverse vers Diverse Scenarios MEC Deployment Future Applications Smart City Smart Home Hazard/Remote Areas Smart Agriculture Digital Twin Holographic Communication AR/VR/XR Metaverse Metaverse Metavers Autonomous Driving Small Data Center Switch BS Traffic Light Satellite Ship Airplane S ll S it h BS T ffi Li ht S t llit Shi Ai l 5G Networks and Beyond 5G Networks and Beyond Smart Factory performance. Its typical algorithms are K-means clustering, principal component analysis, and independent component analysis, which can be used in small cell clustering, heterogeneous network clustering, smart grid user classification, etc. However, in unreliable radio network environments, the classification accuracy decreases, which readily causes slow and inaccurate actions in MEC systems. • Different from the static solutions provided by supervised learning and unsupervised learning, RL gives a constantly evolving intelligent framework [21]-[27]. In RL [29], an agent is enabled to make proper decisions based on frequent interactions with the stochastic and dynamic environment. Based on the Markov models, the RL operates in a feedback mechanism (a closed loop) without the knowledge of input data, where based on the previous and current states, the agent can execute its actions to maximize the reward function. Additionally, the above behaviors, including the states, actions, and rewards, accumulate to generate experiences. The classic RL algorithm is Q-learning, which suffers from the curse of dimensionality caused by the large dimensions of the state-action spaces. Accordingly, upon leveraging the low-dimensional representation for the high-dimensional state-ac...

show abstract

Partial Computation Offloading in NOMA-Assisted Mobile-Edge Computing Systems Using Deep Reinforcement Learning

Cited by 80 publications

References 42 publications

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Joint Multi-domain Resource Allocation and Trajectory Optimization in UAV-assisted M-IoT Networks

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Contact Info

Product

Resources

About