Deep Reinforcement Learning-Based Deterministic Routing and Scheduling for Mixed-Criticality Flows

Yu, Hao; Taleb, Tarik; Zhang, Jiawei

doi:10.1109/tii.2022.3222314

Cited by 10 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, the system is assumed to be fully known, i.e., the list of services and their characteristics are available, and the current state of the network and cloud resources as well as requests and their requirements are being monitored and collected on a regular basis. This could be the case of an industrial environment whereby tasks and communications among robots and devices are pre-planned [32], [33], [34]. Under such scenarios, the following section proposes two methods, B&B-CCRA and WF-CCRA, to solve the problem specified by (1).…”

Section: Fully-informed Methodsmentioning

confidence: 99%

Double Deep Q-Learning-Based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

Shokrnezhad

Taleb

Dazzi

2024

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

Nowadays, as the need for capacity continues to grow, entirely novel services are emerging. A solid cloudnetwork integrated infrastructure is necessary to supply these services in a real-time responsive, and scalable way. Due to their diverse characteristics and limited capacity, communication and computing resources must be collaboratively managed to unleash their full potential. Although several innovative methods have been proposed to orchestrate the resources, most ignored network resources or relaxed the network as a simple graph, focusing only on cloud resources. This paper fills the gap by studying the joint problem of communication and computing resource allocation, dubbed CCRA, including function placement and assignment, traffic prioritization, and path selection considering capacity constraints and quality requirements, to minimize total cost. We formulate the problem as a non-linear programming model and propose two approaches, dubbed B&B-CCRA and WF-CCRA, based on the Branch & Bound and Water-Filling algorithms to solve it when the system is fully known. Then, for partially known systems, a Double Deep Q-Learning (DDQL) architecture is designed. Numerical simulations show that B&B-CCRA optimally solves the problem, whereas WF-CCRA delivers near-optimal solutions in a substantially shorter time. Furthermore, it is demonstrated that DDQL-CCRA obtains near-optimal solutions in the absence of request-specific information.

show abstract

Section: Fully-informed Methodsmentioning

confidence: 99%

Double Deep Q-Learning-Based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

Shokrnezhad

Taleb

Dazzi

2024

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

show abstract

“…Closer to our solution, [11] uses DRL to schedule and route mixed-criticality flows in a Deterministic Networking (DetNet) environment. DetNet operates at the layer 3 whereas TSN operates at layer 2.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning for Time-Aware Shaping (IEEE 802.1Qbv) in Time-Sensitive Networks

Roberty,

Said,

Ridouard

et al. 2023

2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation (ETFA)

View full text Add to dashboard Cite

Industry 4.0 involves the networking of production equipment. This can be achieved thanks to the Time-Sensitive Networking (TSN) set of network standards. However, this new paradigm brings new challenges because TSN features optimization relies on the dynamic characteristics of the underlying communication network (e.g., network topology, routing strategy, critical flows requirements, etc.). This paper focuses on the case of the IEEE 802.1Qbv standard by exploring the applicability of a Deep Reinforcement Learning (DRL) approach in order to reduce the configuration time of the TSN-specific parameters, compared to exact or heuristic methods.

show abstract

“…In contrast to displaying images or playing movies, user interaction, for example, waving a hand right or left to switch content, pressing a finger forward to push a "button" and saying "close" to stop the application, requires real-time communications between user/user or user/server. Therefore, the deterministic networking (DetNet) [3] capabilities, which enable deterministic data delivery, processing and synchronization capacity provided by the underlying networks, become necessary to support the high-quality immersive user experience in real-time.…”

Section: Introductionmentioning

confidence: 99%

Toward 6G-Based Metaverse: Supporting Highly-Dynamic Deterministic Multi-User Extended Reality Services

Yu,

Shokrnezhad,

Taleb

et al. 2023

IEEE Network

Self Cite

View full text Add to dashboard Cite

Metaverse is the concept of a fully immersive and universal virtual space for multiuser interaction, collaboration, and socializing; forming the next evolution of the Internet. Metaverse depends on the convergence of multiple broad technologies that enable eXtended Reality (XR), which is an umbrella term for technologies that lie on the reality-virtuality continuum, namely virtual reality (VR), Augmented Reality (AR), and mixed reality (MR). Compared to streaming volumetric content to a single user, the XR applications that involve multiple users who simultaneously watch the same volumetric content (e.g., VRbased online training/education and multi-user online gaming) are much more attractive. Multi-user XR/Metaverse puts additional demand on the underlying networks, making it more difficult to offer high-quality immersive material in real-time. To cope with this, this article presents a comprehensive system and component design for immersive and seamless multi-user XR experiences. To satisfy the Quality of Experience/Quality of Service (QoE/QoS) requirements and especially stream synchronization requirement in XR collaboration scenarios, we propose an AI-powered deterministic multi-user extended reality resource orchestrator (PRECISENESS) that aims to solve the multi-user XR service provisioning problem. Finally, we demonstrate the performance of our proposed solution in a single-site multi-user XR use case. The obtained results demonstrate that our solution can deliver high-quality immersive XR services.

show abstract

Deep Reinforcement Learning-Based Deterministic Routing and Scheduling for Mixed-Criticality Flows

Cited by 10 publications

References 19 publications

Double Deep Q-Learning-Based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

Double Deep Q-Learning-Based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

Reinforcement Learning for Time-Aware Shaping (IEEE 802.1Qbv) in Time-Sensitive Networks

Toward 6G-Based Metaverse: Supporting Highly-Dynamic Deterministic Multi-User Extended Reality Services

Contact Info

Product

Resources

About