Design and Evaluation of a Self-Learning HTTP Adaptive Video Streaming Client

Abstract

Abstract-HTTP Adaptive Streaming (HAS) is becoming the de facto standard for Over-The-Top (OTT)-based video streaming services such as YouTube and Netflix. By splitting a video into multiple segments of a couple of seconds and encoding each of these at multiple quality levels, HAS allows a video client to dynamically adapt the requested quality during the playout to react to network changes. However, state-of-the-art quality selection heuristics are deterministic and tailored to specific network configurations. Therefore, they are unable to cope with a vast range of highly dynamic network settings. In this letter, a novel Reinforcement Learning (RL)-based HAS client is presented and evaluated. The self-learning HAS client dynamically adapts its behaviour by interacting with the environment to optimize the Quality of Experience (QoE), the quality as perceived by the enduser. The proposed client has been thoroughly evaluated using a network-based simulator and is shown to outperform traditional HAS clients by up to 13% in a mobile network environment.

An HTTP/2 push-based approach for SVC adaptive streaming

“…By averaging the measured bandwidth over a sliding window, fluctuations are smoothed, allowing the client to select a quality level that is sustainable and avoids oscillations. Claeys et al propose to use reinforcement learning, introducing a Q-learning algorithm in the adaptation heuristic that allows to outperform certain deterministic algorithms such as MSS [10]. Focusing on high-RTT networks, Bouten et al propose to use pipelined and parallel download scheduling to reduce the negative impact of the RTT on the user's QoE [11].…”

Cognitive streaming on android devices

“…Each of the contributions is multiply by a constant (C 1 -C 6 ) according to its weighted importance on the overall reward. R t,q (s t , a t ), R t,o (s t , a t ), R t,bf (s t , a t ) and R t,bc (s t , a t ) are based on the reward function derived by [9]. R t,bs (s t , a t ) and R t,ds (s t , a t ) are our own contribution and will be thoroughly discussed.…”

“…To model the oscillation, the authors of [9] defined the length and the depth of the oscillation, where length (OL t ) is the number of video segments since the last observation and depth (OD t ) is the quality difference before and after the oscillation. OL max is the maximum length observed, which means that an oscillation of length higher or equal than OL max will receive an oscillation reward of 0.…”

“…In this regard, Machine Learning and in particular Reinforcement Learning (RL) techniques have proven to fit well in adaptive streaming applications. Examples of this are the StateAction-Reward-State-Action (SARSA) approach of Menkovski et al [8] or the Q-Learning algorithm of Claeys et al [9], [10], which have been studied on desktop computers. Herein, we intend to assess the viability of RL-based adaptive streaming in the context of constrained mobile devices, looking particularly at Android-based terminals.…”

“…Based on the Q-Learning approach presented by Claeys et al [9], we have designed and implemented an Adaptive Video Streaming Application fit for running in Android-based mobile devices. By means of a self-developed wireless experimental test-bed we evaluated the performance of the application under a wide range of conditions.…”