Zero-Cost In-Depth Enforcement of Network Policies for Low-Latency Cloud-Native Systems

Abstract

Packaging applications in containers and managing them dynamically using a cluster orchestrator is the de-facto approach for deployment of cloud-native applications. When containers run inside virtual machines (VMs) to protect infrastructural assets, network policies (NPs) at the container layer and security groups (SGs) at the VM layer provide complementary firewall mechanisms that strengthen defenses against lateral movement of attackers. However, least-privilege NPs at the container layer may not always be consistent with statically defined, over-permissive SGs at the VM layer. This is especially a problem with low-latency configuration of container networking solutions that requires every opened container protocol, port and traffic direction also to be opened at the VM layer. In any post-exploitation scenario where attackers escape from within an already compromised or infected container, such over-permissive SGs do not prevent the attacker from spreading across VMs to find powerful tokens for accessing the cluster orchestrator. In this paper, we introduce GrassHopper (GH), a fast and dynamic cross-layer enforcement approach for NPs, which automatically generates SG configurations from dynamically verified NPs. Given the low-latency context, the design of GH must ensure that dynamically generated SG rules are applied fast before the newly scheduled containers become ready to serve traffic. We evaluate GH on a Kubernetes cluster running on OpenStack. For a wide range of relevant low-latency applications and cluster setups, GH can reduce the network attack surface between VMs at a ratio of 75-to-99% while causing no application level performance overhead with respect to latency, throughput, and CPU utilization.