Dissecting the End-to-end Latency of Interactive Mobile Video Applications

Abstract

In this paper we measure the step-wise latency in the pipeline of three kinds of interactive mobile video applications that are rapidly gaining popularity, namely Remote Graphics Rendering (RGR) of which we focus on mobile cloud gaming, Mobile Augmented Reality (MAR), and Mobile Virtual Reality (MVR). The applications differ from each other by the way in which the user interacts with the application, i.e., video I/O and user controls, but they all share in common the fact that their user experience is highly sensitive to end-to-end latency. Long latency between a user control event and display update renders the application unusable. Hence, understanding the nature and origins of latency of these applications is of paramount importance. We show through extensive measurements that control input and display buffering have a substantial effect on the overall delay. Our results shed light on the latency bottlenecks and the maturity of technology for seamless user experience with these applications.

Profiling Distributed Virtual Environments by Tracing Causality

“…The ability to unify heterogeneous data sources is important for distributed systems, both because some components may not support modification, and also because it is necessary to instrument the real world. Kämäräinen et al [14] and Casiez et al [4] also unified sensor data with internal samples, but their implementations were more tightly coupled than ours.…”

“…Chen et al [5] measured the sources of latency in remote rendering cloud gaming systems by using application hooks to measure the response times of salient events. Kämäräinen et al [14] looked at remote rendering mobile Mobile Virtual Reality (MVR) systems. They comprehensively characterised the end-to-end latency using a multi-modal hardware platform including photo-sensors, touch sensors as well as the ability to time-stamp the client's network traffic.…”

“…For their visualization tool ShiViz, Beschastnikh et al [3] use vector clocks to maintain the logical time between nodes. Kämäräinen et al [14] explicitly synchronise their clocks with an NTP-like protocol. Rueda et al [21] only consider relative times within each client and bypass clock synchronization considerations.…”

CloudVR

“…As a result, we obtained a latency of roughly 100ms and 140ms for the Daydream control with local and remote rendering of the sword, respectively, which highlights the relatively long delays caused by the controller and the mobile device display update pipeline [14]. The end-to-end latency from head movement to updated frame drawn on display was 85-90ms.…”

Hybrid Mobile Augmented Reality: Web-Like Concepts Applied to High Resolution 3d Overlays

“…React Native and the WebView-based Apache Cordova represent the most popular hybrid and interpreted approaches respectively (Biørn-Hansen and Ghinea, 2018). Unlike the latter, the multi-threading technology and the way its components are linked to native UI views endorse React Native with optimized refresh rates, close to 60 fps (Kämäräinen et al, 2016). Cardoso et al (2018) test the applicability of six MAR frameworks in vision-based approach for an outdoor archaeological site and the three prevailing formulate a multiplatform application in the context of Cordova.…”