Stacked Autoencoder-Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks

Abstract

An online resource scheduling framework is proposed for minimizing the sum of weighted task latency for all the mobile users, by optimizing offloading decision, transmission power, and resource allocation in the mobile edge computing (MEC) system. Towards this end, a deep reinforcement learning (DRL) method is proposed to obtain an online resource scheduling policy. Firstly, a related and regularized stacked auto encoder (2r-SAE) with unsupervised learning is proposed to perform data compression and representation for high dimensional channel quality information (CQI) data, which can reduce the state space for DRL. Secondly, we present an adaptive simulated annealing based approach (ASA) as the action search method of DRL, in which an adaptive h-mutation is used to guide the search direction and an adaptive iteration is proposed to enhance the search efficiency during the DRL process. Thirdly, a preserved and prioritized experience replay (2p-ER) is introduced to assist the DRL to train the policy network and find the optimal offloading policy. Numerical results are provided to demonstrate that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computational time compared with existing benchmarks. It also shows that the proposed framework is suitable for resource scheduling problem in large-scale MEC networks, especially in the dynamic environment.Index Terms-Stacked auto encoder, deep reinforcement learning, adaptive simulated annealing, mobile edge computing.

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

“…Table III characterizes the objective function values of the DIRS framework under different distributions of IoTDs. For evaluating the influence of the different position distributions of IoTDs, we define the normalized reward rate (NRR), which is equal to that the inferred reward dividing the optimal reward [44]. In NRR, the inferred reward in the numerator is calculated from the output of the DIRS framework, and the optimal reward in the denominator is obtained from the particle swarm optimization (PSO) algorithm which is always applied to solve large-scale MINLP problems with high quality but low efficiency [44].…”

“…For evaluating the influence of the different position distributions of IoTDs, we define the normalized reward rate (NRR), which is equal to that the inferred reward dividing the optimal reward [44]. In NRR, the inferred reward in the numerator is calculated from the output of the DIRS framework, and the optimal reward in the denominator is obtained from the particle swarm optimization (PSO) algorithm which is always applied to solve large-scale MINLP problems with high quality but low efficiency [44]. From Table III, one can see that the IoTDs with nonuniform distributions (e.g., Gaussian distribution and Lévy distribution) obtain lower objective function values.…”

AI Driven Heterogeneous MEC System with UAV Assistance for Dynamic Environment: Challenges and Solutions

“…Auto-encoder algorithm may be applied. For example, H-MEC could benefit from a pre-training scheme of Stacked Auto-Encoder (SAE) [12] for automatic feature learning. In particular, SAE can be applied to train an attack detection model with a mix of unlabelled normal/attack samples so that the model identifies patterns of attack and normal data by an auto-encoder scheme, this can in turn improve the accuracy of the attack detection model on unseen and mutated attacks.…”

Joint Gateway Selection and Resource Allocation for Cross-Tier Communication in Space-Air-Ground Integrated IoT Networks

“…Simulated annealing (SA) is a probabilistic heuristic search technique based on the annealing process in metallurgy. Thanks to its fast convergence, less parameter, and simplicity, SA has been widely adapted for decision making and optimization in recent years [35]. In this paper, we propose an SA based gateway selection method, i.e., SAGA, to solve P2.1 and obtain the near optimal gateway selection decision, whose details are listed in Algorithm 2.…”