Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Abstract

With the development of edge-cloud computing technologies, distributed data centers (DCs) have been extensively deployed across the global Internet. Since different users/applications have heterogeneous requirements on specific types of ICT resources in distributed DCs, how to optimize such heterogeneous resources under dynamic and even uncertain environments becomes a challenging issue. Traditional approaches are not able to provide effective solutions for multi-dimensional resource allocation that involves the balanced utilization across different resource types in distributed DC environments. This paper presents a reinforcement learning based approach for multidimensional resource allocation (termed as NESRL-MRM) that is able to achieve balanced utilization and availability of resources in dynamic environments. To train NESRL-MRM's agent with sufficiently quick wall-clock time but without the loss of exploration diversity in the search space, a natural evolution strategy (NES) is employed to approximate the gradient of the reward function. To realistically evaluate the performance of NESRL-MRM, our simulation evaluations are based on real-world workload traces from Amazon EC2 and Google datacenters. Our results show that NESRL-MRM is able to achieve significant improvement over the existing approaches in balancing the utilization of multi-dimensional DC resources, which leads to substantially reduced blocking probability of future incoming workload demands.

Resource Allocation With Edge-Cloud Collaborative Traffic Prediction in Integrated Radio and Optical Networks

“…Although there have been various resource allocation methods using the results of traffic prediction, load balancing algorithms and their related service reconfiguration mechanism have been the focus of network research in recent years. Specifically, load balancing resource allocation scheme has been studied in a wireless network, SDN IP network, and transmission control protocol (TCP) network [13], [15], [16], [27], [28], [29]. However, to the best of our knowledge, few load balancing schemes consider the traffic prediction results based on edge-cloud collaboration in integrated radio and optical networks.…”