AR-Based Remote Command and Control Service: Self-Driving Vehicles Use Case

Abstract

The recent technological advances in many fields have significantly contributed to the development of the Advanced Driver Assistance System (ADAS), which in turn will greatly contribute to the flourishing of self-driving vehicles that can operate autonomously in all road scenarios. Until then, keeping the human input in the loop remains vital to either make decisions in unseen situations or approve vehicles' proposed decisions. In this paper, we leverage VR technology to provide remote assistance for self-driving in critical situations. Specifically, we study the delivery of a 360°live stream at high resolution (4K) to a remote operation center for supporting self-driving vehicles' decisions when, for example, merging onto the highway. The 360°video stream will be consumed by a human operator wearing a head-mounted display for increased flexibility, faster control, and an immersive experience. In addition, the 360°s tream is augmented with relevant context data, such as the vehicle's speed and distance to other road objects, in order to increase the human operator's awareness of the vehicle and its surroundings. Depending on the human operator's proximity to the source, the video stream can either be viewed through the cloud or the edge, which further reduces the glass-to-glass latency. Experimental results demonstrate the effectiveness of employing VR technology to remotely command and control self-driving vehicles in critical situations. The results show that a 360°stream at 4K resolution can be delivered in sub-second glass-to-glass latency, which allows the operator to make timely decisions.

VR-Based Immersive Service Management in B5G Mobile Systems: A UAV Command and Control Use Case

“…• Video Streaming 360 • applications are bandwidth intensive and, therefore, optimizing their E2E latency through the network is very challenging [12], [13]. Furthermore, E2E video latency depends strongly on the encoder, the target resolution, bit rate, and most importantly the streaming protocol, such as the real-time messaging protocol (RTMP), real-time streaming protocol (RTSP), or HTTP Live Streaming (also known as HLS) to name a few that are supported by most 360 • cameras [50], [51], [52]. Moreover, compared to the standard streaming systems, interactive VR applications will push the connectivity requirements to their limits due to their strict requirements in terms of capacity, low latency (less than 50-ms latency), as well as consistent Quality of Service (QoS).…”