Self-play learning strategies for resource assignment in Open-RAN networks

Wang, Xiaoyang; Thomas, Jonathan D.; Piechocki, Robert J.; Kapoor, Sahil; Santos-Rodríguez, Raúl; Parekh, Arjun

doi:10.1016/j.comnet.2021.108682

Cited by 12 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table 10, we categorize the reviewed proposals based on the ML technique and the type and number of used algorithms. According to our analysis of the literature review on resource management in RAN slicing, as shown in T-NN 1 Evolutionary Algorithms [65], [66], [68], [69], [67], [70], [72], [71] GA 8 [35], [79], [80], [81], [129], [130], [95] AC 7 [78] MAB 1 [111], [85], [117], [86], [87], [88], [136], [90], [96], [97], [98], [151], [126], [99], [123], [131], [114], [118], [94], [100], [103], [119], [104], [105] DQN 25 [109], [108] Q-Learning 2 Reinforcement Learning [115], [112], [84] Double DQN 3 …”

Section: Discussion and Future Research Directionsmentioning

confidence: 99%

“…The simulation results show that the number of transmitted packets is higher in the DQN method compared to Channel-aware (channel allocation at the beginning of each time-slot), Queue-aware (calculation of queue time or channel usage) and Random (random assignment of each MU to one channel) methods. In [100], the system model is considered as a cluster with DU as the central server and several RUs as the cluster subset. Radio resources in the RU and computational resources in the DU (which perform radio signaling operations) are considered to allocate for slices by considering energy consumption in the DU.…”

Section: ) Resource Sharingmentioning

confidence: 99%

“…To overcome this problem, other RL methods use a neural network to estimate the Q-Value. Compared to the reviewed methods in D-RAN, if the slices need more resources, the resources in the DU or CU can be used in the form of NF to allocate to users of slices in C-RAN and O-RAN based on the requirements of slices such as latency and priority such as [100]. In the reviewed methods for O-RAN (e.g., [94], [100], [124]), NN have been used to overcome the size of Q-Table in Q-learning.…”

Section: ) Lessons Learnedmentioning

confidence: 99%

“…Compared to the reviewed methods in D-RAN, if the slices need more resources, the resources in the DU or CU can be used in the form of NF to allocate to users of slices in C-RAN and O-RAN based on the requirements of slices such as latency and priority such as [100]. In the reviewed methods for O-RAN (e.g., [94], [100], [124]), NN have been used to overcome the size of Q-Table in Q-learning. To increase decision stability in AC [95], we can use a memory buffer to eliminate inefficient decisions or use A2C like [124].…”

Section: ) Lessons Learnedmentioning

confidence: 99%

“…In recent years, the use of distributed methods (such as transfer learning and federated learning) in ML has been considered due to the common challenges in centralized methods [79], [80], [81], [82], [84], [85], [86], [87], [88], [89], [90], [91], [92], [83], [93], [94], [95], [110], [111], [112], [113], [114], [115], [116], [117], [118], [136] ✓ ✓ ---- [96], [97], [98], [99], [100], [103], [104], [105], [123], [124], [135] [107], [108], [109], [126], [127], [128], [35], [129], [130] such as centralization, security, privacy, and time and computational complexity. In distributed methods, a ML mode...…”

Section: Distributed Learningmentioning

confidence: 99%

See 4 more Smart Citations

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

et al. 2022

View full text Add to dashboard Cite

One of the key foundations of 5 th Generation (5G) and beyond 5G (B5G) networks is network slicing, in which the network is partitioned into several separated logical networks, taking into account the requirements of diverse applications. In this context, resource management is of great importance to instantiate and operate network slices and meet their performance and functional requirements. Resource management in Radio Access Networks (RANs) is associated with a range of challenges due to network dynamics and the specific requirements of each application while ensuring performance isolation. In this paper, we present a survey on state-of-the-art works that employ Machine Learning (ML) techniques in RAN slicing. We begin by reviewing the challenges, then we review the existing papers on resource management in a comprehensive manner, and classify the papers based on the used ML algorithm, the addressed challenges, and the type of allocated resources. We evaluate the maturity of current methods and state a number of open challenges and some solutions to address these challenges in RAN resource management.

show abstract

Section: Discussion and Future Research Directionsmentioning

confidence: 99%

Section: ) Resource Sharingmentioning

confidence: 99%

Section: ) Lessons Learnedmentioning

confidence: 99%

Section: ) Lessons Learnedmentioning

confidence: 99%

Section: Distributed Learningmentioning

confidence: 99%

See 3 more Smart Citations

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

et al. 2022

View full text Add to dashboard Cite

show abstract

Beyond games: a systematic review of neural Monte Carlo tree search applications

Kemmerling,

Lütticke,

Schmitt

2023

Appl Intell

View full text Add to dashboard Cite

The advent of AlphaGo and its successors marked the beginning of a new paradigm in playing games using artificial intelligence. This was achieved by combining Monte Carlo tree search, a planning procedure, and deep learning. While the impact on the domain of games has been undeniable, it is less clear how useful similar approaches are in applications beyond games and how they need to be adapted from the original methodology. We perform a systematic literature review of peer-reviewed articles detailing the application of neural Monte Carlo tree search methods in domains other than games. Our goal is to systematically assess how such methods are structured in practice and if their success can be extended to other domains. We find applications in a variety of domains, many distinct ways of guiding the tree search using learned policy and value functions, and various training methods. Our review maps the current landscape of algorithms in the family of neural monte carlo tree search as they are applied to practical problems, which is a first step towards a more principled way of designing such algorithms for specific problems and their requirements.

show abstract

RLOps: Development Life-Cycle of Reinforcement Learning Aided Open RAN

Thomas

Wang

et al. 2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

Radio access network (RAN) technologies continue to evolve, with Open RAN gaining the most recent momentum. In the O-RAN specifications, the RAN intelligent controllers (RICs) are softwaredefined orchestration and automation functions for the intelligent management of RAN. This article introduces principles for machine learning (ML), in particular, reinforcement learning (RL) applications in the O-RAN stack. Furthermore, we review the state-of-the-art research in wireless networks and cast it onto the RAN framework and the hierarchy of the O-RAN architecture. We provide a taxonomy for the challenges faced by ML/RL models throughout the development life-cycle: from the system specification to production deployment (data acquisition, model design, testing and management, etc.). To address the challenges, we integrate a set of existing MLOps principles with unique characteristics when RL agents are considered. This paper discusses a systematic model development, testing and validation lifecycle, termed: RLOps. We discuss fundamental parts of RLOps, which include: model specification, development, production environment serving, operations monitoring and safety/security. Based on these principles, we propose the best practices for RLOps to achieve an automated and reproducible model development process. At last, a holistic data analytics platform rooted in the O-RAN deployment is designed and implemented, aiming to embrace and fulfil the aforementioned principles and best practices of RLOps.

show abstract

Self-play learning strategies for resource assignment in Open-RAN networks

Cited by 12 publications

References 30 publications

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

Applications of Machine Learning in Resource Management for RAN-Slicing in 5G and Beyond Networks: A Survey

Beyond games: a systematic review of neural Monte Carlo tree search applications

RLOps: Development Life-Cycle of Reinforcement Learning Aided Open RAN

Contact Info

Product

Resources

About