Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing

Zhao, Junhui; Kong, Ming; Li, Qiuping; Sun, Xiaoke

doi:10.1109/access.2019.2963051

Cited by 55 publications

(18 citation statements)

References 34 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even if there are no prior works dealing with DRL for the computation of VM placement strategies in datacenters, the literature includes many works witnessing its flexibility to support the use of cloud computing in many emerging and challenging scenarios. Among the others, we mention the dynamic activation of fog nodes in green radio access networks (Sun et al 2018), the offload of cloud devices (Chen et al 2018), and the task distribution in vehicular networks (Zhao et al 2020).…”

Section: Related Workmentioning

confidence: 99%

Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacenters

et al. 2020

View full text Add to dashboard Cite

The ubiquitous diffusion of cloud computing requires suitable management policies to face the workload while guaranteeing quality constraints and mitigating costs. The typical trade-off is between the used power and the adherence to a service-level metric subscribed by customers. To this aim, a possible idea is to use an optimization-based placement mechanism to select the servers where to deploy virtual machines. Unfortunately, high packing factors could lead to performance and security issues, e.g., virtual machines can compete for hardware resources or collude to leak data. Therefore, we introduce a multi-objective approach to compute optimal placement strategies considering different goals, such as the impact of hardware outages, the power required by the datacenter, and the performance perceived by users. Placement strategies are found by using a deep reinforcement learning framework to select the best placement heuristic for each virtual machine composing the workload. Results indicate that our method outperforms bin packing heuristics widely used in the literature when considering either synthetic or real workloads.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacenters

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Therefore, with the benefits of the DRL approach, it is taken to deal with this challenge. According to article [74], the wireless transmission bandwidth and the computation power contribution among vehicles are investigated. A DRL approach-based contract theory is proposed to manage these resources to reduce complexity and to avoid decision collisions among vehicles.…”

Section: Supervised Learning-assisted Techniquesmentioning

confidence: 99%

DRL‐Based Intelligent Resource Allocation for Diverse QoS in 5G and toward 6G Vehicular Networks: A Comprehensive Survey

Nguyen

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The vehicular network is taking great attention from both academia and industry to enable the intelligent transportation system (ITS), autonomous driving, and smart cities. The system provides extremely dynamic features due to the fast mobile characteristics. While the number of different applications in the vehicular network is growing fast, the quality of service (QoS) in the 5G vehicular network becomes diverse. One of the most stringent requirements in the vehicular network is a safety-critical real-time system. To guarantee low-latency and other diverse QoS requirements, wireless network resources should be effectively utilized and allocated among vehicles, such as computation power in cloud, fog, and edge servers; spectrum at roadside units (RSUs); and base stations (BSs). Historically, optimization problems have mostly been investigated to formulate resource allocation and are solved by mathematical computation methods. However, the optimization problems are usually nonconvex and hard to be solved. Recently, machine learning (ML) is a powerful technique to cope with the complexity in computation and has capability to cope with big data and data analysis in the heterogeneous vehicular network. In this paper, an overview of resource allocation in the 5G vehicular network is represented with the support of traditional optimization and advanced ML approaches, especially a deep reinforcement learning (DRL) method. In addition, a federated deep reinforcement learning- (FDRL-) based vehicular communication is proposed. The challenges, open issues, and future research directions for 5G and toward 6G vehicular networks, are discussed. A multiaccess edge computing assisted by network slicing and a distributed federated learning (FL) technique is analyzed. A FDRL-based UAV-assisted vehicular communication is discussed to point out the future research directions for the networks.

show abstract

“…Thus they are sufficient to perform computations and communications by themselves. The concept of vehicular fog computing (VFC) has been proposed in [38][39][40][41]. Generally, the vehicular fog network is an integrated network consisting a group of vehicles and multiple edge servers.…”

Section: B Vehicles As Computation Resourcesmentioning

confidence: 99%

“…Here, vehicles are used to offload computations from the nearby vehicles, which can be considered to serve an internal system. In [40], the authors propose a novel contractbased incentive mechanism to make vehicles join vehicular fog computing. Distributed deep reinforcement learning is used to offload computations to the participating vehicles, and task offloading method based on the queueing model is also proposed to avoid decision collisions in multivehicles task offloading.…”

Section: B Vehicles As Computation Resourcesmentioning

confidence: 99%

A Hierarchical Vehicular-Based Architecture for Vehicular Networks: A Case Study on Computation Offloading

et al. 2020

View full text Add to dashboard Cite

In order to realize an intelligent transportation system (ITS) which will provide smooth urban traffic, autonomous driving, accurate route navigation, etc., enormous computations need to be migrated from cloud centers to edge nodes, especially for the services requiring stringent latency. In addition to base stations and road side units (RSUs), vehicles can be alteratively considered as a kind of computation resources. In this paper, a hierarchical vehicular-based architecture which consists of cloud centers and vehicles is investigated. Computation offloading performance in the hierarchical architecture is also studied. In specific, the main components in vehicular networks and their characteristics on communication and computations are presented firstly. Several communication techniques that are essential in enabling computation offloading among these components are then discussed. Secondly, a hierarchical vehicular-based architecture, which integrates the main components, is constructed. Thirdly, a case study on computation offloading in the proposed architecture is conducted. In the concerned scenario, the computation offloading problem is modelled as a multi-dimensional multiple knapsack problem (MMKP). Two algorithms are investigated, among which, the first algorithm is a greedy heuristic method providing a sub-optimal solution with a low computational complexity. The second algorithm is a modified branch and bound (B&B) method, which can obtain the best solution with a high computational complexity. Numerical results are also presented to verify the performance of the two algorithms. It can be demonstrated that the proposed architecture can migrate more computations from cloud centers to vehicular nodes, when the computations require more communication resources.

show abstract

Contract-Based Computing Resource Management via Deep Reinforcement Learning in Vehicular Fog Computing

Cited by 55 publications

References 34 publications

Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacenters

Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacenters

DRL‐Based Intelligent Resource Allocation for Diverse QoS in 5G and toward 6G Vehicular Networks: A Comprehensive Survey

A Hierarchical Vehicular-Based Architecture for Vehicular Networks: A Case Study on Computation Offloading

Contact Info

Product

Resources

About