Empirical evaluation of latency-sensitive application performance in the cloud

Abstract

Cloud computing platforms enable users to rent computing and storage resources on-demand to run their networked applications and employ virtualization to multiplex virtual servers belonging to different customers on a shared set of servers. In this paper, we empirically evaluate the efficacy of cloud platforms for running latency-sensitive multimedia applications. Since multiple virtual machines running disparate applications from independent users may share a physical server, our study focuses on whether dynamically varying background load from such applications can interfere with the performance seen by latency-sensitive tasks. We first conduct a series of experiments on Amazon's EC2 system to quantify the CPU, disk, and network jitter and throughput fluctuations seen over a period of several days. We then turn to a laboratory-based cloud and systematically introduce different levels of background load and study the ability to isolate applications under different settings of the underlying resource control mechanisms. We use a combination of micro-benchmarks and two realworld applications-the Doom 3 game server and Apple's Darwin Streaming Server-for our experimental evaluation. Our results reveal that the jitter and the throughput seen by a latency-sensitive application can indeed degrade due to background load from other virtual machines. The degree of interference varies from resource to resource and is the most pronounced for disk-bound latencysensitive tasks, which can degrade by nearly 75% under sustained background load. We also find that careful configuration of the resource control mechanisms within the virtualization layer can mitigate, but not eliminate, this interference.

A hybrid edge-cloud architecture for reducing on-demand gaming latency

“…Past studies [21,22,32] have found that players begin to notice a delay of 100 ms [33]. However, 20 ms of this latency may be attributed to playout and processing delay [13]. Therefore, 80 ms is the threshold above which network latency begins to appreciably affect user experience.…”

“…However, CDN edge servers are generally built from commodity hardware that have relative weak computational capabilities and often lack GPUs. Moreover, although these servers are designed to serve thousands of simultaneous end-users, current virtualization technologies do not enable many instances of a game engine to run concurrently on the same physical machine [13]. Additionally, GPU requirements for games are sufficiently high such that running more than one game is currently not feasible.…”

“…Consequently, we select 80 ms as a network latency target since previous works [21,33,41] indicate that 100 ms is the latency threshold that is required for realism and acceptable gameplay for action games. Because at least 20 ms can be attributed to playout and processing delay [13], network latency can account for up to 80 ms of the total latency. We select 40 ms as a stricter requirement for games that require a significant amount of processing or multiplayer coordination.…”

“…For the purpose of gaming, the work in [13] shows that processing delay is affected by the amount of computational resources provisioned by the provider (which could be a cloud or CDN provider), and this delay ranges from 10 ms to more than 30 ms. The provider is responsible for the processing delay.…”

Distributed Neural Networks for Internet of Things: The Big-Little Approach

“…Currently, Cloud computing is most often the natural choice to perform this data processing, benefiting from the huge compute power and scalability of current Cloud infrastructure [14]. However, the Cloud is not a silver bullet solution, as a network connection to a Cloud datacenter often suffers from high and variable latency, as well as a limited upload bandwidth [2]. Moreover, extensive Cloud processing incurs considerable cost.…”