Improved decentralized Q-learning algorithm for interference reduction in LTE-femtocells

Simsek, Meryem; Czylwik, Andreas; Galindo-Serrano, Ana; Giupponi, Lorenza

doi:10.1109/wiad.2011.5983301

Cited by 52 publications

(31 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where α = 0.5 is the player's willingness to learn from its environment, λ = 0.9 is the discount factor, and s ′ is the next state [34], [35]. Hereby, the agent's previous knowledge about the state-action pair (s, a) is represented by the first term in (9).…”

Section: B Q-learning Based Time-domain Icicmentioning

confidence: 99%

Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets

Simsek

Bennis

Güvenç

2015

IEEE Trans. Veh. Technol.

Self Cite

102

View full text Add to dashboard Cite

Abstract-In this article, we focus on inter-cell interference coordination (ICIC) techniques in heterogeneous network (HetNet) deployments, whereby macro-and picocells autonomously optimize their downlink transmissions, with loose coordination. We model this strategic coexistence as a multi-agent system, aiming at joint interference management and cell association. Using tools from Reinforcement Learning (RL), agents (i.e., macro-and picocells) sense their environment, and self-adapt based on local information so as to maximize their network performance. Specifically, we explore both time-and frequency domain ICIC scenarios, and propose a two-level RL formulation. Here, picocells learn their optimal cell range expansion (CRE) bias and transmit power allocation, as well as appropriate frequency bands for multi-flow transmissions, in which a user equipment (UE) can be simultaneously served by two or more base stations (BSs) from macro-and pico-layers. To substantiate our theoretical findings, Long Term Evolution Advanced (LTE-A) based system level simulations are carried out in which our proposed approaches are compared with a number of baseline approaches, such as resource partitioning (RP), static CRE, and single-flow Carrier Aggregation (CA). Our proposed solutions yield substantial gains up to 125% compared to static ICIC approaches in terms of average UE throughput in the timedomain. In the frequency-domain our proposed solutions yield gains up to 240% in terms of cell-edge UE throughput.

show abstract

Section: B Q-learning Based Time-domain Icicmentioning

confidence: 99%

Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets

Simsek

Bennis

Güvenç

2015

IEEE Trans. Veh. Technol.

Self Cite

102

View full text Add to dashboard Cite

show abstract

“…Moreover, an improved effective initialization procedure was provided to overcome slow convergence the drawback of Q-learning. The RL approach based on Q-Learning framework has been also proposed to handle channel sharing between small cells deployed in a macro cell [41]. Automatic Neighbour Relation Management concerns the updates of neighbour cell relationships to facilitate easy handovers between base stations.…”

Section: Automatic Configuration Of Initial Radio Transmission Paramementioning

confidence: 99%

Application of Cognitive Techniques to Network Management and Control

Kukliński

Wytrębowicz

Tantar

2014

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Abstract. This paper describes the latest communications technologies emphasizing the need of dynamic network control and real-time management operations. It is advocated that many such operations can profit from cognitive learning based techniques that could drive many management or control operations. In that context a short overview of selected networking approaches like 3GPP Self Organizing Networks, Autonomic Network Management and Software-Defined Networking, with some references to existing cognitive approaches is given.

show abstract

“…To overcome this drawback, we proposed an initialization procedure, which shows major convergence/performance enhancement [6]. Visiting one state for the first time not only the Q-value of a single state-action pair is updated, but estimates for the cost function for all other possible actions of the current state are added, so that the Qtable is rapidly initialized.…”

Section: Decentralized Q-learning With Improved Initializationmentioning

confidence: 99%

On Interference Management Techniques in LTE Heterogeneous Networks

Simsek

Czylwik

Bennis

2012

2012 21st International Conference on Computer Communications and Networks (ICCCN)

Self Cite

View full text Add to dashboard Cite

Autonomous interference management solutions for inter-cell interference coordination (ICIC) are of utmost importance. In this paper, the coexistence between macro and small cells is studied whereby different ICIC techniques pertaining to different deployment and information assumptions are evaluated. Inspired from Evolutionary Game Theory (EGT), decentralized strategies are devised, in which small cell Base Stations (BSs) exchange information through a central controller, and adapt their strategies based on instantaneous and average payoffs of the small cell population. In contrast, when distributed operation is aimed at, using tools from Reinforcement Learning (RL) small cells learn by interacting with their environment through trialsand-errors, and autonomously optimize their strategies based on a mere feedback. In particular, we compare the performance of decentralized Q-learning, Fuzzy Q-learning, improved Qlearning and expertness-based Q-learning procedures. Finally, the overall performance of the network in terms of average peruser data throughput and convergence are carried out in an LTE-A system level simulator.Index Terms-LTE heterogeneous networks, small cells, interference management, reinforcement learning, game theory.

show abstract

Improved decentralized Q-learning algorithm for interference reduction in LTE-femtocells

Cited by 52 publications

References 3 publications

Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets

Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets

Application of Cognitive Techniques to Network Management and Control

On Interference Management Techniques in LTE Heterogeneous Networks

Contact Info

Product

Resources

About