TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks

Li, Rongpeng; Zhao, Zhifeng; Chen, Xianfu; Palicot, Jacques; Zhang, Honggang

doi:10.1109/twc.2014.022014.130840

Cited by 131 publications

(115 citation statements)

References 24 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…In addition to game theory, machine learning has been considered as an effective tool in solving different network problems in 5G [15], [16]. RL is one of the most powerful tools for policy control and intelligent decision making [5], which has been widely adopted in wireless communications [17]- [19]. Recently, a number of works have applied RL to solve the intelligent resource management and decision making problem in D2D underlay networks [20]- [27].…”

Section: Arxiv:191209302v1 [Csni] 18 Dec 2019mentioning

confidence: 99%

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Guo

2020

IEEE Trans. Veh. Technol.

114

View full text Add to dashboard Cite

Device-to-device (D2D) communication underlay cellular networks is a promising technique to improve spectrum efficiency. In this situation, D2D transmission may cause severe interference to both the cellular and other D2D links, which imposes a great technical challenge to spectrum allocation. Existing centralized schemes require global information, which causes a large signaling overhead. While existing distributed schemes requires frequent information exchange among D2D users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multiagent deep reinforcement learning is proposed, named multiagent actor critic (MAAC). MAAC shares global historical states, actions and policies during centralized training, requires no signal interaction during execution and utilizes cooperation among users to further optimize system performance. Moreover, in order to decrease the computing complexity of the training, we further propose the neighbor-agent actor critic (NAAC) based on the neighbor users' historical information for centralized training. The simulation results show that the proposed MAAC and NAAC can effectively reduce the outage probability of cellular links, greatly improve the sum rate of D2D links and converge quickly.Index Terms-Device-to-device (D2D) communications, multiagent deep reinforcement learning, spectrum allocation.

show abstract

Section: Arxiv:191209302v1 [Csni] 18 Dec 2019mentioning

confidence: 99%

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Guo

2020

IEEE Trans. Veh. Technol.

114

View full text Add to dashboard Cite

show abstract

“…Therefore, the total number of flows in the system is equal to

\sum_{b \in B_{ON}} \frac{L_{b}}{1 - L_{b}}

, which is proportional to the expected delay. () Thus, with a low system load on average, the UEs experience less delay …”

Section: System Modelmentioning

confidence: 99%

“…, which is proportional to the expected delay. 17,18 Thus, with a low system load on average, the UEs experience less delay. 19…”

Section: Figurementioning

confidence: 99%

Distributed ON/OFF switching and dynamic channel allocation: Decreasing complexity and improving energy efficiency

Arani

Mehbodniya

Omidi

et al. 2017

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Heterogeneous network (HetNet) has emerged as a promising and viable approach to deliver higher‐rate data services. In HetNets, switching OFF base stations (BSs) is known as an effective solution to reduce the energy consumption of networks and thus enables the “greener” networks concept. In this paper, we investigate the impact of the number of energy‐saving modes for BSs on improving energy efficiency. We assume that BSs are able to switch among several modes, ie, different transmission power levels. We apply a no‐regret learning algorithm to choose the BS's operation mode while modeling the BS's operation under dynamic traffic with a Markov process. As a result, we face a trade‐off between energy savings and computational complexity. Furthermore, we use a distributed channel allocation algorithm based on the average cochannel interference power. Simulation results show that energy consumption reduces with increased number of modes. However, increasing the number of modes beyond a certain limit will not have a significant effect on energy efficiency.

show abstract

“…More specifically, different criteria (e.g., traffic load [11], user spatial distribution [12] or networkimpact [13]) are considered in order to identify the optimal BS switching off strategy, guaranteeing a certain level of user satisfaction in the network. In the same context, a very recent interesting approach is the application of reinforcement learning schemes that cope with the dynamic nature of the traffic load in current cellular networks [14] in order to overcome an important limitation of the existing works, which rely on past predefined traffic patterns based on the network history.…”

Section: Base Station Switching Off and Infrastructure Sharingmentioning

confidence: 99%

Energy-efficient infrastructure sharing in multi-operator mobile networks

et al. 2015

View full text Add to dashboard Cite

Network infrastructure sharing and base station (BS) switching off mechanisms have been recently introduced as promising solutions towards energy and cost reduction in cellular networks.Although these techniques are usually studied independently, their combination offers new alternatives to mobile network operators (MNOs) for serving their users and could potentially provide them with additional benefits. In this article, we introduce the concept of intra-cell roamingbased infrastructure sharing, where the MNOs may switch off their BSs and roam their traffic to active BSs operated by other MNOs in the same cell. Motivated by the coexistence of multiple operators in the same area, we present possible network deployments and architectures in current and future cellular scenarios, discussing their particular characteristics. In addition, we propose an innovative distributed game theoretic BS switching off scheme, employing an integrated cost function that takes into account all the different cases for a given operator to serve its own traffic (i.e., through active BSs of neighboring cells or exploiting intra-cell roaming-based infrastructure sharing). Finally, we demonstrate some indicative simulation results in realistic scenarios to quantify the potential energy and financial benefits that our proposed scheme offers to the MNOs

show abstract

TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks

Cited by 131 publications

References 24 publications

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Distributed ON/OFF switching and dynamic channel allocation: Decreasing complexity and improving energy efficiency

Energy-efficient infrastructure sharing in multi-operator mobile networks

Contact Info

Product

Resources

About