A Safe Reinforcement Learning Architecture for Antenna Tilt Optimisation

Aumayr, Erik; Feghhi, Saman; Vannella, Filippo; Håkim, Ezeddin Al; Iakovidis, Grigorios

doi:10.1109/pimrc50174.2021.9569387

Cited by 6 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SAC is a model-free state-of-the-art RL algorithm (Haarnoja et al, 2018a,b). DQN (Mnih et al, 2013) is a well-known discrete action-space model-free method used in several previous works on RL for antenna down-tilt control (Vannella et al, 2021;Bouton et al, 2021;Aumayr et al, 2021). For DQN, the action space is changed to update the tilt by increments of {−1°, 0°, 1°}.…”

Section: Methodsmentioning

confidence: 99%

“…In the coverage and capacity optimization problem, the goal is to control the tilt of the antenna such that all users have good coverage, good signal quality, and that many users can send and receive data at the same time. When adjusting the parameters of the antennas, these three quantities will be affected, and the down-tilt angle w is the one of the most influential parameters (Buenestado et al, 2017;Vannella et al, 2021;Dreifuerst et al, 2021). Fig.…”

Section: Problem Formulationmentioning

confidence: 99%

“…There is a trade-off between coverage, capacity and quality when controlling the antenna tilt Buenestado et al (2017);Vannella et al (2021). This is illustrated in Fig.…”

Section: C2 Policy Learning Stepmentioning

confidence: 99%

“…Interestingly, recent work in this domain has shown that RL agents can outperform traditional rule-based policies. One such example is the coverage and capacity optimization problem, where the agent controls the down-tilt of antennas in order to optimize the KPIs coverage, capacity and quality (Vannella et al, 2021;Buenestado et al, 2017). The existence of rule-based strategies for this problem is a strong asset that can be used in combination with RL algorithms.…”

Section: Introductionmentioning

confidence: 99%

“…Previous works have addressed the problem of antenna tilt control with RL by using standard algorithms, such as DQN Mnih et al (2013), or focused on the multi-agent aspect (Farooq et al, 2019;Bouton et al, 2021;Dandanov et al, 2017) without considering sample efficiency. Another body of literature has investigated safe reinforcement learning approaches (Nikou et al, 2021;Aumayr et al, 2021). For instance, Vannella et al used a rule-based policy as a behavioral policy to gather data from the environment.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Model Based Residual Policy Learning with Applications to Antenna Control

Möllerstedt¹,

Russo²,

Bouton³

2022

Preprint

View full text Add to dashboard Cite

Non-differentiable controllers and rule-based policies are widely used for controlling real systems such as robots and telecommunication networks. In this paper, we present a practical reinforcement learning method which improves upon such existing policies with a model-based approach for better sample efficiency. Our method significantly outperforms state-of-the-art model-based methods, in terms of sample efficiency, on several widely used robotic benchmark tasks. We also demonstrate the effectiveness of our approach on a control problem in the telecommunications domain, where model-based methods have not previously been explored. Experimental results indicate that a strong initial performance can be achieved and combined with improved sample efficiency. We further motivate the design of our algorithm with a theoretical lower bound on the performance.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

“…There is a trade-off between coverage, capacity and quality when controlling the antenna tilt Buenestado et al (2017);Vannella et al (2021). This is illustrated in Fig.…”

Section: C2 Policy Learning Stepmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Model Based Residual Policy Learning with Applications to Antenna Control

Möllerstedt¹,

Russo²,

Bouton³

2022

Preprint

View full text Add to dashboard Cite

show abstract

Imperative Action Masking for Safe Exploration in Reinforcement Learning

Dey,

Bhat,

Dasgupta

et al. 2023

Explainable and Transparent AI and Multi-Agent Systems

View full text Add to dashboard Cite

Network Parameter Control in Cellular Networks through Graph-Based Multi-Agent Constrained Reinforcement Learning

Forsberg,

Nikou,

Feljan

et al. 2023

2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)

View full text Add to dashboard Cite

Cellular networks are growing in complexity at increasing speed and the geographical locations in which they are deployed in are getting denser. Traditional control methods fall short in providing a scalable and dynamic way of adapting the network to new conditions. Distributed multiagent reinforcement learning successfully addresses scalability problems seen in centralized approaches. The question of achieving learning with constraint satisfaction in distributed systems is still left unanswered in the state-of-the-art. In this work, we aim to perform distributed multi-agent constrained reinforcement learning in order to learn a policy online while satisfying imposed constraints. We use a coordination graph to model the interactions between agents and decompose the global value function. A conservative safety critic is trained in parallel to evaluate the safety of proposed actions. Our method allows for separate training of both the critic and the value network independently of each other, and hence offers flexibility in how and when to train the different models. The results are compared to a baseline using no safety critic. Simulations show that the agents succeed in learning a policy that can satisfy the constraints, while still maximizing the objective.

show abstract

A Safe Reinforcement Learning Architecture for Antenna Tilt Optimisation

Cited by 6 publications

References 8 publications

Model Based Residual Policy Learning with Applications to Antenna Control

Model Based Residual Policy Learning with Applications to Antenna Control

Imperative Action Masking for Safe Exploration in Reinforcement Learning

Network Parameter Control in Cellular Networks through Graph-Based Multi-Agent Constrained Reinforcement Learning

Contact Info

Product

Resources

About