Self-play Learning Strategies for Resource Assignment in Open-RAN Networks

Wang, Xiaoyang; Thomas, Jonathan D.; Piechocki, Robert J.; Kapoor, Sahil; Santos-Rodríguez, Raúl; Parekh, Arjun

doi:10.48550/arxiv.2103.02649

Cited by 2 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conduct an additional set of experiments comparing against (Wang et al, 2021), which adopts the RR algorithm , where L π denotes the length of the minimum bounding box that encompass all packed boxes at the end of episodes; as in Sec.3.4, we train our agent attend2pack with the utility reward r u but compare to the results of (Wang et al, 2021) using their measure r L . The packing visualizations and the final testing results are shown in Fig.…”

Section: Appendix: Additional Ablation Study On the Sequence Policymentioning

confidence: 99%

Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention

Zhang¹,

Zi²,

Ge³

2021

Preprint

View full text Add to dashboard Cite

This paper seeks to tackle the bin packing problem (BPP) through a learning perspective. Building on self-attention-based encoding and deep reinforcement learning algorithms, we propose a new end-to-end learning model for this task of interest. By decomposing the combinatorial action space, as well as utilizing a new training technique denoted as prioritized oversampling, which is a general scheme to speed up on-policy learning, we achieve state-of-the-art performance in a range of experimental settings. Moreover, although the proposed approach attend2pack targets offline-BPP, we strip our method down to the strict online-BPP setting where it is also able to achieve state-of-the-art performance. With a set of ablation studies as well as comparisons against a range of previous works, we hope to offer as a valid baseline approach to this field of study.

show abstract

Section: Appendix: Additional Ablation Study On the Sequence Policymentioning

confidence: 99%

Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention

Zhang¹,

Zi²,

Ge³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…[28] shows the scheme of effective energy using through RL dynamic function splitting. [29] demonstrates an application of using RL for computational resource allocation between RUs and DUs, which has the potential to reduce power consumption significantly.…”

Section: O-cloudmentioning

confidence: 99%

RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN

Li¹,

Thomas²,

Wang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Radio access network (RAN) technologies continue to witness massive growth, with Open RAN gaining the most recent momentum. In the O-RAN specifications, the RAN intelligent controller (RIC) serves as an automation host. This article introduces principles for machine learning (ML), in particular, reinforcement learning (RL) relevant for the O-RAN stack. Furthermore, we review state-of-the-art research in wireless networks and cast it onto the RAN framework and the hierarchy of the O-RAN architecture. We provide a taxonomy of the challenges faced by ML/RL models throughout the development life-cycle: from the system specification to production deployment (data acquisition, model design, testing and management, etc.). To address the challenges, we integrate a set of existing MLOps principles with unique characteristics when RL agents are considered. This paper discusses a systematic life-cycle model development, testing and validation pipeline, termed: RLOps. We discuss all fundamental parts of RLOps, which include: model specification, development and distillation, production environment serving, operations monitoring, safety/security and data engineering platform. Based on these principles, we propose the best practices for RLOps to achieve an automated and reproducible model development process.

show abstract

Self-play Learning Strategies for Resource Assignment in Open-RAN Networks

Cited by 2 publications

References 23 publications

Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention

Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention

RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN

Contact Info

Product

Resources

About