Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis

Kouchaki, Mohammadreza; Marojevic, Vuk

doi:10.1109/gcwkshps56602.2022.10008713

Cited by 15 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The growing interest in the O-RAN architecture has also led the scientific community to focus its attention on the implementation of xApps and rApps. Several papers in the literature implement xApps [14,41,42] and rApps [43] or both working cooperatively [44,45] that can be onboarded and tested on the RICs for studying different use cases such as network slicing, orchestration and RAN management, and security. Compared to these works, in this paper, we focus on the TS use case which is a problem usually for large-scale scenarios with hundreds of UEs.…”

Section: Dual Connectivity and Traffic Steeringmentioning

confidence: 99%

“…Compared to these works, in this paper, we focus on the TS use case which is a problem usually for large-scale scenarios with hundreds of UEs. In [42], Kouchaki et al illustrate the step-by-step design, development, and testing of an AI-based resource allocation xApp for the near-RT RIC of the O-RAN architecture without focusing on the actual results of the AI-scheme proposed, but it is not clear if their online training can be performed on a real RAN. In contrast, our work focuses more on the xApp performances and the creation of a novel framework to ease the study of RL applied to O-RAN.…”

Section: Dual Connectivity and Traffic Steeringmentioning

confidence: 99%

See 1 more Smart Citation

Programmable and Customized Intelligence for Traffic Steering in 5G Networks Using Open RAN Architectures

Lacava

Polese

Sivaraj³

et al. 2024

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

5G and beyond mobile networks will support heterogeneous use cases at an unprecedented scale, thus demanding automated control and optimization of network functionalities customized to the needs of individual users. Such fine-grained control of the Radio Access Network (RAN) is not possible with the current cellular architecture. To fill this gap, the Open RAN paradigm and its specification introduce an "open" architecture with abstractions that enable closed-loop control and provide data-driven, and intelligent optimization of the RAN at the userlevel. This is obtained through custom RAN control applications (i.e., xApps) deployed on near-real-time RAN Intelligent Controller (near-RT RIC) at the edge of the network. Despite these premises, as of today the research community lacks a sandbox to build data-driven xApps, and create large-scale datasets for effective Artificial Intelligence (AI) training. In this paper, we address this by introducing ns-O-RAN, a software framework that integrates a real-world, production-grade near-RT RIC with a 3GPP-based simulated environment on ns-3, enabling at the same time the development of xApps and automated large-scale data collection and testing of Deep Reinforcement Learning (DRL)driven control policies for the optimization at the user-level. In addition, we propose the first user-specific O-RAN Traffic Steering (TS) intelligent handover framework. It uses Random Ensemble Mixture (REM), a Conservative Q-learning (CQL) algorithm, combined with a state-ofthe-art Convolutional Neural Network (CNN) architecture, to optimally assign a serving base station to each user in the network. Our TS xApp, trained with more than 40 million data points collected by ns-O-RAN, runs on the near-RT RIC and controls the ns-O-RAN base stations. We evaluate the performance on a large-scale deployment with up to 126 users with 8 base stations, showing that the xApp-based handover improves throughput and spectral efficiency by an average of 50% over traditional handover heuristics, with less mobility overhead.

show abstract

Section: Dual Connectivity and Traffic Steeringmentioning

confidence: 99%

Section: Dual Connectivity and Traffic Steeringmentioning

confidence: 99%

Programmable and Customized Intelligence for Traffic Steering in 5G Networks Using Open RAN Architectures

Lacava

Polese

Sivaraj³

et al. 2024

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

show abstract

“…Currently, it seems a trend to solve the power allocation problem for wireless networks by adopting deep reinforcement learning methods [2] [3] [4]. We notice that solutions for UDSCNs in previous work had no consideration of coupling problem, which means that components such as local and central controllers of the framework are tightly depended and have low robustness.…”

Section: Introductionmentioning

confidence: 99%

Low-Coupling Policy Optimization Framework for Power Allocation in Ultra-Dense Small-Cell Networks

Chen,

Liu,

Huang

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) methods have emerged as a feasible solution for addressing the power resource allocation problem in ultra-dense small-cell networks (UDSCNs). In this paper, we propose a novel actor-critic-based low-coupling policy optimization (LCPO) framework. Our framework aims to achieve practicality by employing a design that consists of training and execution modules with low coupling. By adopting policy optimization methods, including advantage actor-critic (A2C) and proximal policy optimization (PPO) with statedependent exploration (SDE) technique, LCPO demonstrates stable performance. In this study, we define the research problem of power resource allocation in UDSCNs and present the mathematical algorithm employed in the LCPO framework. We compare the performance of LCPO with other algorithms, such as deep deterministic policy gradient (DDPG) and fractional programming (FP) algorithms. Through extensive simulations, our proposed LCPO framework outperforms DDPG and FP algorithms in terms of both performance and execution time. Furthermore, to provide an up-to-date overview of the current state-of-the-art, we incorporate recent research papers in the field. The inclusion of these papers enhances the relevance of our study and allows readers to gain insights into the latest advancements in power resource allocation in UDSCNs. The results of our research highlight the effectiveness of the LCPO framework in addressing the power resource allocation problem in UDSCNs. The proposed framework offers superior performance compared to existing algorithms, making it a promising solution for optimizing power allocation in UDSCNs.INDEX TERMS Actor-critic, power allocation, policy optimization, small-cell networks.

show abstract

Integration of RIC and xApps for Open -Radio Access Network for Performance Optimization

Sadashivappa,

Palepu

2023

2023 7th International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS)

View full text Add to dashboard Cite

Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis

Cited by 15 publications

References 10 publications

Programmable and Customized Intelligence for Traffic Steering in 5G Networks Using Open RAN Architectures

Programmable and Customized Intelligence for Traffic Steering in 5G Networks Using Open RAN Architectures

Low-Coupling Policy Optimization Framework for Power Allocation in Ultra-Dense Small-Cell Networks

Integration of RIC and xApps for Open -Radio Access Network for Performance Optimization

Contact Info

Product

Resources

About