Instability in Geo-Distributed Kubernetes Federation: Causes and Mitigation

Abstract

As resources in geo-distributed environments are typically located in remote sites characterized by high latency and intermittent network connectivity, delays and transient network failures are common between the management layer and the remote resources. In this paper, we show that delays and transient network failures coupled with static configuration, including the default configuration parameter values, can lead to instability of application deployments in Kubernetes Federation, making applications unavailable for long periods of time. Leveraging on the benefits of configuration tuning, we propose a feedback controller to dynamically adjust the concerned configuration parameter to improve the stability of application deployments without slowing down the detection of hard failures. We show the effectiveness of our approach in a geo-distributed setup across five sites of Grid'5000, bringing system stability from 83-92% with no controller to 99.5-100% using the controller.

Scheduling Multi-Component Applications Across Federated Edge Clusters With Phare

“…), and Ψ j,v = min r∈Γ (Ψ r j,v ), (10) extracting the minimum affinity value among the existing resources, hence considering the most conservative scenario.…”

“…Indeed, these solutions have been designed for centralized data centers, with guarantees of computing and network resources, and are not designed to identify suitable microservice placement considering their communication patterns. Therefore, they fail to scale on geographically distributed edge-like infrastructures seamlessly, specifically when dealing with nodes that are geographically spread over highlatency WANs [10]- [12].…”

Edge Workloads Monitoring and Failover: a StarlingX-Based Testbed Implementation and Measurement Study

“…V. In Sec. VI, we discuss the experimental setup and Several DCI implementations involving IoT deployments 183 emphasised the DCI benefits to Edge computing use 184 cases [6], [7], [8], [9], [10], [11], [12], [13]. The imple-185 mented DCIs in these studies, as shown in Fig.…”

“…• Third, despite the efforts toward federated Kubernetes clusters, recent implementations show that Kubernetes federation controllers cannot scale to a sufficient size for Edge computing use cases [9] and can lead to Edge workload deployment instability [10]. Furthermore, Kubernetes-based distributed workload deployments (e.g., multi-cluster and multi-cloud) necessitate the use of additional cluster management tools (e.g., Open Cluster Management 6 ) to automate cluster registration, work distribution, and dynamic policy and workload placement.…”