A Q-learning strategy for federation of 5G services

Antevski, Kiril; Martín‐Pérez, Jorge; García‐Saavedra, Andrés; Bernardos, Carlos J.; Xi, Li; Baranda, Jorge; Mangues‐Bafalluy, Josep; Martínez, Ricardo; Vettori, Luca

doi:10.1109/icc40277.2020.9149082

Cited by 9 publications

(12 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within the 5Growth project (and its predecessor 5G-TRANSFORMER), several static and dynamic federation mechanisms have already been proposed, validated and evaluated in [33] This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.…”

Section: ) Communication Service Levelmentioning

confidence: 99%

Multi-Domain Solutions for the Deployment of Private 5G Networks

Li¹,

Guimarães

Landi

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Private 5G networks has become a popular choice of various vertical industries to build dedicated and secure wireless networks in industry environments to deploy their services with enhanced service flexibility and device connectivity to foster industry digitalization. This article proposes multiple multi-domain solutions to deploy private 5G networks for vertical industries across their local premises and interconnecting them with the public networks. Such scenarios open up a new market segment for various stakeholders, and break the current operators' business and service provisioning models. This, in turn, demands new interactions among the different stakeholders across their administrative domains. To this aim, three distinct levels of multi-domain solutions for deploying vertical's 5G private networks are proposed in this work, which can support interactions at different layers among various stakeholders, allowing for different levels of service exposure and control. Building on a set of industry verticals (comprising Industry 4.0, Transportation and Energy), different deployment models are analyzed and the proposed multidomain solutions are applied. These solutions are implemented and validated through two proof-of-concept prototypes integrating a 5G private network platform (5Growth platform) with public ones. These solutions are being implemented in three vertical pilots conducted with real industry verticals. The obtained results demonstrated the feasibility of the proposed multi-domain solutions applied at the three layers of the system enabling various levels of interactions among the different stakeholders. The achieved end-to-end service instantiation time across multiple domains is in the range of minutes, where the delay impact caused by the resultant multi-domain interactions is considerably low. The proposed multi-domain approaches offer generic solutions and standard interfaces to support the different private network deployment models.

show abstract

Section: ) Communication Service Levelmentioning

confidence: 99%

Multi-Domain Solutions for the Deployment of Private 5G Networks

Li¹,

Guimarães

Landi

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…As the radius of operation of both selected use cases spans across multiple geographic locations, vertical services extend outside the local domain using any of the aforementioned interactions, to achieve full connectivity that guarantees the performance KPIs and SLAs. Reinforcement Learning, and specifically Q-Learning, can be applied to achieve profit maximization for the vertical service deployment, relying on data such as the available resources in the local and peering providers, and price of a service [12]. The complete 5G solution for both use cases is depicted in Figure 2.…”

Section: B 5growth-based Enhancements To Selected Use Casesmentioning

confidence: 99%

Public and Non-Public Network Integration for 5Growth Industry 4.0 Use Cases

Guimarães

Papagianni

et al. 2021

IEEE Commun. Mag.

Self Cite

View full text Add to dashboard Cite

5G is playing a paramount role in the digital transformation of the industrial sector, offering high bandwidth, reliable and low latency wireless connectivity to meet the stringent and critical performance requirements of manufacturing processes. This work analyzes the applicability of 5G technologies as key enablers to support, enhance and even enable novel advances in Industry 4.0. It proposes a complete 5G solution for two real world Industry 4.0 use cases related to metrology and quality control. Such solution uses 5Growth to ease and automate the management of vertical services over a Software-Defined Network (SDN) and Network Function Virtualization (NFV) based 5G mobile transport and computing infrastructure, and to aid the integration of the verticals' private 5G network with the public network. Finally, a validation campaign (i) assesses the applicability of the proposed solution to support the performance requirements (especially latency and user data rate) of the selected use cases, and (ii) evaluates its efficiency regarding vertical service setup time across different domains in less than 3min.

show abstract

“…In [19], an initial attempt to apply Q-Learning to the service federation problem was presented. In this paper, we go beyond by formulating the problem as an MDP to obtain the optimal solution and we propose a new average-reward learning algorithm that outperforms Q-Learning under all evaluated scenarios and whose performance is not as sensitive to parameter tuning (e.g., discount factor) as Q-learning.…”

Section: Related Workmentioning

confidence: 99%

R-Learning-Based Admission Control for Service Federation in Multi-domain 5G Networks

Bakhshi¹,

Mangues‐Bafalluy²

2021

2021 IEEE Global Communications Conference (GLOBECOM)

Self Cite

View full text Add to dashboard Cite

Network service federation in 5G/B5G networks enables service providers to extend service offering by collaborating with peering providers. Realizing this vision requires interoperability among providers towards end-to-end service orchestration across multiple administrative domains. Smart admission control is fundamental to make such extended offering profitable. Without prior knowledge of service requests, the admission controller (AC) either determines the domain to deploy the demand or rejects it to maximize the long-term average profit. In this paper, we first obtain the optimal AC policy by formulating the problem as a Markov decision process, which is solved through the policy iteration method. This provides the theoretical performance bound under the assumption of known arrival and departure rates of demands. Then, for practical solutions to be deployed in real systems, where the rates are not known, we apply the Q-Learning and R-Learning algorithms to approximate the optimal policy. The extensive simulation results show that learning approaches outperform the greedy policy and are capable of getting close to optimal performance. More specifically, R-learning always outperformed the rest of practical solutions and achieved an optimality gap of 3-5% independent of the system configuration, while Q-Learning showed lower performance and depended on discount factor tuning.

show abstract

A Q-learning strategy for federation of 5G services

Cited by 9 publications

References 3 publications

Multi-Domain Solutions for the Deployment of Private 5G Networks

Multi-Domain Solutions for the Deployment of Private 5G Networks

Public and Non-Public Network Integration for 5Growth Industry 4.0 Use Cases

R-Learning-Based Admission Control for Service Federation in Multi-domain 5G Networks

Contact Info

Product

Resources

About