Work-in-Progress: NVIDIA GPU Scheduling Details in Virtualized Environments

Abstract

Modern automotive grade embedded platforms feature high performance Graphics Processing Units (GPUs) to support the massively parallel processing power needed for next-generation autonomous driving applications. Hence, a GPU scheduling approach with strong Real-Time guarantees is needed. While previous research e orts focused on reverse engineering the GPU ecosystem in order to understand and control GPU scheduling on NVIDIA platforms, we provide an in depth explanation of the NVIDIA standard approach to GPU application scheduling on a Drive PX platform. Then, we discuss how a privileged scheduling server can be used to enforce arbitrary scheduling policies in a virtualized environment.

GPU Devices for Safety-Critical Systems: A Survey

“…-Device hardware architecture and component-specific (e.g., memory [70,43,68,101,109,104], cache [69,87,154,96,97,194]) -Exclusive access policies [96,97,194] -Software virtualization [84,114,141,41,174,112] -WCET analysis [48,29,30,85,33,16,90,89,88] -Execution time variability reduction and management [18,41,42,67,82,84,88,142,144,200,101,70,68,109,43,141,147,47,81,24] -Temporal diagnostics …”

“…For example, the execution time variability of the Apollo AD software framework with Linux can be up to 21x (maximum) and 6.1x on average, with "highly variable timing behavior and arbitrary distributions" of internal software modules [16]. In order to reduce this variability and enable the real-time execution of algorithms, diverse techniques are proposed, e.g., scheduling [18,41,42,67,82,84,88,142,144,200], memory scheduling and arbitration mechanisms [101,70,68,109,43,141,147], fine-grained CPU-GPU command offloading with Vulkan [47], and (devicespecific) real-time neural networks software execution adaptation with a WCET performance model of the device [81]. Finally, additional case studies of interest that analyze the real-time characteristics of software applications implemented in GPUs and CUDA are: automotive sign detection application [146,201] and power train application [79] that uses the GPU programming patterns described by Barbieri et al [24].…”

Performance Analysis and Optimization Opportunities for NVIDIA Automotive GPUs

“…While conclusions reached by those works are highly valuable for an efficient use of hardware, they do not provide any insight on how to optimize hardware design. Finally, some works target task scheduling on GPUs for an efficient use of hardware resources, minimizing their timespan while respecting their deadlines [17], [10], [9].…”

Graphic Interfaces in ADAS

“…As highlighted in previous sections, ADAS embedded platforms are commonly virtualized through a hypervisor. Inter-VM communication must be secure and access to GPU computing power must be deterministically scheduled [5,12]. A single scheduling policy must be active in a given time instant, although dynamic policy reconfigurations are allowed.…”