Cognitive Cellular Networks: A Q-Learning Framework for Self-Organizing Networks

Mwanje, Stephen S.; Schmelz, Lars Christoph; Mitschele‐Thiel, Andreas

doi:10.1109/tnsm.2016.2522080

Cited by 82 publications

(54 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As such, these techniques can be considered as effective ways to enable intelligent handover management. References [15]- [17] described reinforcement learning-based handover parameter optimisation methods. In Reference [15], the authors proposed a handover parameter tuning method that effectively detects handover events and minimises false handover triggers.…”

Section: Reinforcement Based Handover Triggering Optimisation Methodsmentioning

confidence: 99%

Intelligent Handover Triggering Mechanism in 5G Ultra-Dense Networks Via Clustering-Based Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

Ultra-dense networks (UDNs) are considered as key 5G technologies. They provide mobile users a high transmission rate and efficient radio resource management. However, UDNs lead to the dense deployment of small base stations (BSs) that can cause stronger interference and subsequently increase the handover management complexity. At present, the conventional handover triggering mechanism of user equipment (UE) is only designed for macro mobility and thus could result in negative effects such as frequent handovers, ping-pong handovers, and handover failures on the handover process of UE at UDNs. These effects degrade the overall network performance. In addition, a massive number of BSs significantly increase the network maintenance system workload. To address these issues, this paper proposes an intelligent handover triggering mechanism for UE based on Q-learning frameworks and subtractive clustering techniques. The input metrics are first converted to state vectors by subtractive clustering, which can improve the efficiency and effectiveness of the training process.Afterward, the Q-learning framework learns the optimal handover triggering policy from the environment. The trained Q table is deployed to UE to trigger the handover process. The simulation results demonstrate that the proposed method can ensure the stronger mobility robustness of UE that is improved by 60%-90% compared to the conventional approach with respect to the number of handovers, ping-ping handover rate, and handover failure rate while maintaining other key performance indicators (KPIs), that is, a relatively high level of throughput and network latency. In addition, through integration with subtractive clustering, the proposed mechanism is further improved by an average of 20% in terms of all the evaluated KPIs.

show abstract

Section: Reinforcement Based Handover Triggering Optimisation Methodsmentioning

confidence: 99%

Intelligent Handover Triggering Mechanism in 5G Ultra-Dense Networks Via Clustering-Based Reinforcement Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Depending on the mobility observed in each cell, the algorithm applies a certain action and receives a penalty or reward. The solution in [197] also relies on QL. This time, however, the authors consider both MRO and Mobility Load Balancing (MLB) use cases.…”

Section: E Handover Parameters Optimizationmentioning

confidence: 99%

A Survey of Machine Learning Techniques Applied to Self-Organizing Cellular Networks

Klaine

Imran

Onireti

et al. 2017

IEEE Commun. Surv. Tutorials

393

200

View full text Add to dashboard Cite

Abstract-In this paper, a survey of the literature of the past fifteen years involving Machine Learning (ML) algorithms applied to self organizing cellular networks is performed. In order for future networks to overcome the current limitations and address the issues of current cellular systems, it is clear that more intelligence needs to be deployed, so that a fully autonomous and flexible network can be enabled. This paper focuses on the learning perspective of Self Organizing Networks (SON) solutions and provides, not only an overview of the most common ML techniques encountered in cellular networks, but also manages to classify each paper in terms of its learning solution, while also giving some examples. The authors also classify each paper in terms of its self-organizing use-case and discuss how each proposed solution performed. In addition, a comparison between the most commonly found ML algorithms in terms of certain SON metrics is performed and general guidelines on when to choose each ML algorithm for each SON function are proposed. Lastly, this work also provides future research directions and new paradigms that the use of more robust and intelligent algorithms, together with data gathered by operators, can bring to the cellular networks domain and fully enable the concept of SON in the near future.

show abstract

“…Generally, the classic off-policy method optimizes Algorithm 1 Load-driven Clustering Based on K-means 1: Input: 2: t; ρ t i ; x i ; number of clusters H; number of SBSs N . 3: Initialization: 4: Calculate the stage-averaged loads according to (17). 5: Rank the stage-averaged loads to obtain C list .…”

Section: Self-organized Drl-based Load Balancingmentioning

confidence: 99%

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach

Wang

et al. 2019

IEEE Internet Things J.

View full text Add to dashboard Cite

In this paper, we propose a deep reinforcement learning (DRL) based mobility load balancing (MLB) algorithm along with a two-layer architecture to solve the large-scale load balancing problem for ultra-dense networks (UDNs). Our contribution is three-fold. First, this work proposes a two-layer architecture to solve the large-scale load balancing problem in a self-organized manner. The proposed architecture can alleviate the global traffic variations by dynamically grouping small cells into self-organized clusters according to their historical loads, and further adapt to local traffic variations through intra-cluster load balancing afterwards. Second, for the intra-cluster load balancing, this paper proposes an off-policy DRL-based MLB algorithm to autonomously learn the optimal MLB policy under an asynchronous parallel learning framework, without any prior knowledge assumed over the underlying UDN environments. Moreover, the algorithm enables joint exploration with multiple behavior policies, such that the traditional MLB methods can be used to guide the learning process thereby improving the learning efficiency and stability. Third, this work proposes an offlineevaluation based safeguard mechanism to ensure that the online system can always operate with the optimal and well-trained MLB policy, which not only stabilizes the online performance but also enables the exploration beyond current policies to make full use of machine learning in a safe way. Empirical results verify that the proposed framework outperforms the existing MLB methods in general UDN environments featured with irregular network topologies, coupled interferences, and random user movements, in terms of the load balancing performance.

show abstract

Cognitive Cellular Networks: A Q-Learning Framework for Self-Organizing Networks

Cited by 82 publications

References 21 publications

Intelligent Handover Triggering Mechanism in 5G Ultra-Dense Networks Via Clustering-Based Reinforcement Learning

Intelligent Handover Triggering Mechanism in 5G Ultra-Dense Networks Via Clustering-Based Reinforcement Learning

A Survey of Machine Learning Techniques Applied to Self-Organizing Cellular Networks

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach

Contact Info

Product

Resources

About