Cell handover has been considered as one of the most challenging issues in LTE-A macro-femtocell networks, due to the ad hoc deployment nature of Femto Base Stations (FBSs). In this paper, our goal is to achieve seamless handover of Mobile Te rminal (MT) among different cells while improving the system throughput. First, we formulate the handover decision and channel allocation problem as a Markov Decision Problem (MDP) whose objective is to maximize the total discounted expected reward per connection over an inOilite horizon. The MDP formulation takes various system parameters into account such as MT moving velocity, MT buffer size, cell switching cost and call dropping penalty. Then, we propose an algorithm based on Q-Iearning techniques to obtain the optimal cell handover and channel allocation policy. Simulation results show that our proposed algorithm is effective, and can maximize the expected total reward and reduce the average number of handovers.Index Terms Handover decision, Markov decision process, Femtocell, Q-Iearning.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.