Sensor and Actuator Latency during Teleoperation of Automated Vehicles

Georg, Jean-Michael; Feiler, Johannes; Hoffmann, Simon; Diermeyer, Frank

doi:10.1109/iv47402.2020.9304802

Cited by 30 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Commands are written to, and feedback read from the vehicle CAN bus through a dSpace Autobox, also connected to the vehicle PC via Ethernet. The end-to-end delay, the so-called glass-to-glass (G2G) latency [25], of a 40 Hz, 520p video feed, transmitted over a wired connection and displayed to the operator on a gaming monitor, operating at 144 Hz, is approximately 104 ms. A thorough assessment and comparisons of the latency for different configurations within the same system are provided in [26].…”

Section: A Passenger Vehiclementioning

confidence: 99%

Open Source Software for Teleoperated Driving

Schimpe¹,

Feiler²,

Hoffmann³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Teleoperation allows a human operator to remotely interact with and control a mobile robot in a dangerous or inaccessible area. Besides well-known applications such as space exploration or search and rescue operations, the application of teleoperation in the area of automated driving, i.e., teleoperated driving (ToD), is becoming more popular. Instead of an in-vehicle human fallback driver, a remote operator can connect to the vehicle using cellular networks and resolve situations that are beyond the automated vehicle (AV)'s operational design domain. Teleoperation of AVs, and unmanned ground vehicles in general, introduces different problems, which are the focus of ongoing research. This paper presents an open source ToD software stack, which was developed for the purpose of carrying out this research. As shown in three demonstrations, the software stack can be deployed with minor overheads to control various vehicle systems remotely.

show abstract

Section: A Passenger Vehiclementioning

confidence: 99%

Open Source Software for Teleoperated Driving

Schimpe¹,

Feiler²,

Hoffmann³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…First of all, the teleoperation is subject to latency due to delays in system components and transmission of the data via a mobile network. However, with advances in computational power, sensor and actuator technologies, delays can be reduced significantly [1]. In addition, novel mobile network standards promise even greater reduction [2].…”

Section: A Teleoperated Drivingmentioning

confidence: 99%

“…Several hardware and software design choices for the ToD system were driven by the goal to minimize the age of the information that is being displayed to the operator. An extensive and thorough analysis of hardware components and their latencies is presented in [1]. The design choices include high camera frame rates and, despite lower compression rates, the use of the faster H.264 over H.265 [21].…”

Section: Experimental System Setupmentioning

confidence: 99%

“…With the described parameter settings and other hardware component choices, such as gaming monitors with minimal input lag and high refresh rates, the latency of the video streaming system can be reduced effectively. With a wired connection, the end-to-end latency, measured from an event happening in front of the camera until it is being displayed on the operator monitor, is below 120 ms for the three frontmounted cameras [1]. Although this paper does not explicitly aim at optimizing latency, these characteristics of the given ToD system are described here, as they significantly influence the obtained rate-quality models that are presented in the following section.…”

Section: Experimental System Setupmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Video Configuration and Bitrate Allocation for Teleoperated Vehicles

Schimpe¹,

Hoffmann²,

Diermeyer³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Vehicles with autonomous driving capabilities are present on public streets. However, edge cases remain that still require a human in-vehicle driver. Assuming the vehicle manages to come to a safe state in an automated fashion, teleoperated driving technology enables a human to resolve the situation remotely by a control interface connected via a mobile network. While this is a promising solution, it also introduces technical challenges, one of them being the necessity to transmit video data of multiple cameras from the vehicle to the human operator. In this paper, an adaptive video streaming framework specifically designed for teleoperated driving is proposed and demonstrated. The framework enables automatic reconfiguration of the video streams of the multi-camera system at runtime. Predictions of variable transmission service quality are taken into account. With the objective to maximize visual quality, the framework uses so-called rate-quality models to dynamically allocate bitrates and select resolution scaling factors. Results from deploying the proposed framework on an actual teleoperated driving system are presented.

show abstract

“…Similarly, in the case of a long-haul truck, remote driving might even solve a more immediate issue; the current lack of truck drivers is causing logistics to find new ways of solving the issue. Remote driving could provide a possible solution for this (Gardner, 2021). Similar benefits as with mining apply in this field as well: multiple remote drivers can turn by turn take over control of one truck, without having to take into account the normal legal driving and resting times that come with one driver per one truck normally.…”

Section: Introductionmentioning

confidence: 99%

Design and Evaluation of Remote Driving Architecture on 4G and 5G Mobile Networks

Ouden

Smagt³

et al. 2022

Front. Future Transp.

View full text Add to dashboard Cite

Despite the progress in the development of automated vehicles in the last decade, reaching the level of reliability required at large-scale deployment at an economical price and combined with safety requirements is still a long road ahead. In certain use cases, such as automated shuttles and taxis, where there is no longer even a steering wheel and pedals required, remote driving could be implemented to bridge this gap; a remote operator can take control of the vehicle in situations where it is too difficult for an automated system to determine the next actions. In logistics, it could even be implemented to solve already more pressing issues such as shortage of truck drivers, by providing more flexible working conditions and less standstill time of the truck. An important aspect of remote driving is the connection between the remote station and the vehicle. With the current roll-out of 5G mobile technology in many countries throughout the world, the implementation of remote driving comes closer to large-scale deployment. 5G could be a potential game-changer in the deployment of this technology. In this work, we examine the remote driving application and network-level performance of remote driving on a recently deployed sub-6-GHz commercial 5G stand-alone (SA) mobile network. It evaluates the influence of the 5G architecture, such as mobile edge computing (MEC) integration, local breakout, and latency on the application performance of remote driving. We describe the design, development (based on Hardware-in-the-Loop simulations), and performance evaluation of a remote driving solution, tested on both 5G and 4G mobile SA networks using two different vehicles and two different remote stations. Two test cases have been defined to evaluate the application and network performance and are evaluated based on position accuracy, relative reaction times, and distance perception. Results show the performance of the network to be sufficient for remote driving applications at relatively low speeds (<40 km/h). Network latencies compared with 4G have dropped to half. A strong correlation between latency and remote driving performance is not clearly seen and requires further evaluation taking into account the influence of the user interface.

show abstract

Sensor and Actuator Latency during Teleoperation of Automated Vehicles

Cited by 30 publications

References 14 publications

Open Source Software for Teleoperated Driving

Open Source Software for Teleoperated Driving

Adaptive Video Configuration and Bitrate Allocation for Teleoperated Vehicles

Design and Evaluation of Remote Driving Architecture on 4G and 5G Mobile Networks

Contact Info

Product

Resources

About