Multi-Agent Team Learning in Virtualized Open Radio Access Networks (O-RAN)

Abstract

Starting from the concept of the Cloud Radio Access Network (C-RAN), continuing with the virtual Radio Access Network (vRAN) and most recently with the Open RAN (O-RAN) initiative, Radio Access Network (RAN) architectures have significantly evolved in the past decade. In the last few years, the wireless industry has witnessed a strong trend towards disaggregated, virtualized and open RANs, with numerous tests and deployments worldwide. One unique aspect that motivates this paper is the availability of new opportunities that arise from using machine learning, more specifically multi-agent team learning (MATL), to optimize the RAN in a closed-loop where the complexity of disaggregation and virtualization makes well-known Self-Organized Networking (SON) solutions inadequate. In our view, Multi-Agent Systems (MASs) with MATL can play an essential role in the orchestration of O-RAN controllers, i.e., near-real-time and non-real-time RAN Intelligent Controllers (RIC). In this article, we first provide an overview of the landscape in RAN disaggregation, virtualization and O-RAN, then we present the state-of-the-art research in multi-agent systems and team learning as well as their application to O-RAN. We present a case study for team learning where agents are two distinct xApps: power allocation and radio resource allocation. We demonstrate how team learning can enhance network performance when team learning is used instead of individual learning agents. Finally, we identify challenges and open issues to provide a roadmap for researchers in the area of MATL based O-RAN optimization.

RLOps: Development Life-Cycle of Reinforcement Learning Aided Open RAN

“…[30] CU-DU resource assignment Neural MCTS 5.70% -12.95% at resource utilisation efficiency Iturria-Rivera et.al. [31] Power and radio resource allocation Multi-agent DRL Higher energy utilization and throughput Mungari [32] Radio resource management -Dynamic resource allocation based on traffic flow Zhang et.al. [33] Power and radio resource allocation Team [2] described some of the AI-based deployment targets, such as service level agreement (SLA) assured 5G RAN slice, context-based dynamic handover management for vehicle-to-everything (V2X), traffic steering, and flight path based dynamic unmanned aerial vehicle (UAV) resource allocation etc., while we believe the potential of AI-enabled O-RAN is far more than that.…”

Designing a 6G Testbed for Location: Use Cases, Challenges, Enablers and Requirements

“…In this respect, Open Radio Access Networks (Open RAN) [22], [23] is a major breakthrough that is already being used in 5G networks and will continue to be a central feature in 6G, enabling the easy integration of software components from different vendors and speedy creation of new services and functions. Software Defined Networks (SDN [24]) will allow the definition of a dynamic architecture, that can reconfigure the network quickly and adapt it to changes in context (such as changes in traffic, variations in user behaviors, the Deep Network Slicing [17], [25] functionality, the occurrence of catastrophic events, etc.). In this context, 6G will develop a new concept of network operations that will be based on dynamic resource allocation (both in terms of network structure and network functions) for optimizing the general network efficiency [26].…”

Hierarchical Reinforcement Learning for RIS-Assisted Energy-Efficient RAN

“…In addition, such joint control capabilities of handling timescale differences are critical to many existing platforms such as O-RAN and associated control loops; i.e. non-RT (with decision horizon in hours) and near-RT RIC (with decision horizon in seconds or milliseconds) [14].…”