Coexistence Scheme for Uncoordinated LTE and WiFi Networks Using Experience Replay Based Q-Learning

Abstract

Nowadays, broadband applications that use the licensed spectrum of the cellular network are growing fast. For this reason, Long-Term Evolution-Unlicensed (LTE-U) technology is expected to offload its traffic to the unlicensed spectrum. However, LTE-U transmissions have to coexist with the existing WiFi networks. Most existing coexistence schemes consider coordinated LTE-U and WiFi networks where there is a central coordinator that communicates traffic demand of the co-located networks. However, such a method of WiFi traffic estimation raises the complexity, traffic overhead, and reaction time of the coexistence schemes. In this article, we propose Experience Replay (ER) and Reward selective Experience Replay (RER) based Q-learning techniques as a solution for the coexistence of uncoordinated LTE-U and WiFi networks. In the proposed schemes, the LTE-U deploys a WiFi saturation sensing model to estimate the traffic demand of co-located WiFi networks. We also made a performance comparison between the proposed schemes and other rule-based and Q-learning based coexistence schemes implemented in non-coordinated LTE-U and WiFi networks. The simulation results show that the RER Q-learning scheme converges faster than the ER Q-learning scheme. The RER Q-learning scheme also gives 19.1% and 5.2% enhancement in aggregated throughput and 16.4% and 10.9% enhancement in fairness performance as compared to the rule-based and Q-learning coexistence schemes, respectively.

Technology recognition and traffic characterization for wireless technologies in ITS band

“…Technology recognition can be used to minimize this implementation complexity as it enables an active node to sense the wireless environment and identify the traffic statistics of other co-located networks [32]. The extracted information can be used to formulate coexistence decisions [33] without signaling exchange and without the need for a complex receiver.…”

A Multiarmed Bandit Approach for LTE-U/Wi-Fi Coexistence in a Multicell Scenario

“…For example, the authors of [18] overview and compare some of the access mechanisms mentioned above, while the authors of [19] show through an experimental analysis that a single LTE parameter can greatly impact WiFi's performance. Moreover, we want to highlight the studies [6,[20][21][22][23], which have made important contributions to the field. We highlight below their most promising remarks:…”

Wi-Fi Meets ML: A Survey on Improving IEEE 802.11 Performance With Machine Learning

“…Additionally, for prediction applications, the preprocessing of a dataset is crucial to obtain generalized solutions. • Girmay et al [239] point out that QL has been overused (at least in the area of network coexistence) since "Qlearning is not an efficient solution for problems with dynamic environments". Experience replay is proposed as an alternative.…”