Proactive QoE Provisioning in Heterogeneous Access Networks using Hidden Markov Models and Reinforcement Learning

Abstract

Quality of Experience (QoE) provisioning in heterogeneous access networks (HANs) can be achieved via handoffs. The current approaches for QoE-aware handoffs either lack the availability of a network path probing method or lack the availability of efficient methods for QoE prediction. Further, the current approaches do not explore the benefits of proactive QoE-aware handoffs such that user's QoE is maximized by learning from past network conditions and by actions taken by the mobile device regarding handoffs. In this paper, our contributions are two-fold. First, we propose, develop and validate a novel method for QoE prediction based on passive probing. Our method is based on hidden Markov models and Multi-homed Mobility Management Protocol which eliminates the need for additional probe packets for QoE prediction. It achieves the average QoE prediction accuracy of 97%. Second, we propose, develop and validate a novel reinforcement learning based method for proactive QoE-aware handoffs. We show that our method outperforms existing approaches by reducing the number of vertical handoffs by 60.65% while maintaining high QoE levels and by extending crucial functionality such as passive probing mechanisms.