Mitigating Unnecessary Throttling in Linux CFS Bandwidth Control

Abstract

An operating system needs to fairly allocate shared hardware resources among different applications, and Linux uses Completely Fair Scheduler (CFS) to achieve this goal. To ensure fairness, CFS implements bandwidth control that sets the maximum limit on the resources that a process can use. Setting the upper limit helps CFS to discover badly behaving applications and hard cap them to limit the overall damage to the system and other applications, thereby improving fairness.We observe that, in an effort to ensure fairness, the bandwidth control can unnecessarily throttle processes which results in poor application performance. We investigate the root cause of this limitation and discover that the CPU runtime accounting mechanism of bandwidth control, which tracks if a process has reached it maximum allocated limit, is responsible for it. We also find that the overhead of fair resource allocation in bandwidth control can become significantly high due to the way it is implemented. We propose mechanisms to reduce throttling as well as the overhead. Our experimental results show that the proposed techniques are able to eliminate nearly all throttling, thus providing up to 12% performance gain. Also, our approach reduces the bandwidth control overhead by up to 24x.

QoS for best‐effort batch jobs in container‐based cloud

“…The containers, pods, and QoS classes of Kubernetes form a hierarchical tree structure, which are mapped into a hierarchy of task groups of Linux 18 as shown in Figure 1. The Linux cgroup specifies the CPU resources reserved by a task group via the period, quota, and shares attributes.…”

“…For example, Kubernetes provides requests and limits interfaces to reserve resources. These interfaces are ultimately translated into the period, quota, and shares attributes of control group (cgroup) of Linux 18 . Thus, we have to deal with mapping between resource reservation interfaces and attributes of the OS‐level scheduler.…”