Voilà: Tail-Latency-Aware Fog Application Replicas Autoscaler

Abstract

Latency-sensitive fog computing applications may use replication both to scale their capacity and to place application instances as close as possible to their end users. In such geo-distributed environments, a good replica placement should maintain the tail network latency between end-user devices and their closest replica within acceptable bounds while avoiding overloaded replicas. When facing non-stationary workloads it is essential to dynamically adjust the number and locations of a fog application's replicas. We propose Voilà, a tail-latencyaware auto-scaler integrated in the Kubernetes orchestration system. Voilà maintains a fine-grained view of the volumes of traffic generated from different user locations, and uses simple yet highly-effective procedures to maintain suitable application resources in terms of size and location.

Model-based Stream Processing Auto-scaling in Geo-Distributed Environments

“…We rely on the popular open-source Kubernetes (k8s) container orchestrator. Although it was initially designed for cluster and cloud environments, it is now being adapted to handle fog computing scenarios as well [29]- [32]. Kubernetes provides a variety of mechanisms to simplify the deployment, scaling and management of containerized applications in distributed environments by managing their complete life-cycle.…”

mck8s: An orchestration platform for geo-distributed multi-cluster environments

“…As resources in a fog computing environment are geographically distributed with heterogeneity in resources and network characteristics and location, the problem of resource management has been revisited by a number of works. Some works focus on optimizing the placement of jobs and services [17], [18], whereas others address the joint problem of placement and autoscaling [2], [19]- [21]. Similarly, in this work, we address the issues of placement and autoscaling together.…”

“…k8s has fully demonstrated its ability to efficiently orchestrate the resources within a single cluster. However, it does not implement any notion of resource location and therefore does not allow its users to control the location of resources assigned to run their applications [2]. Kubernetes Federations (KubeFed) extend k8s with an explicit notion of multi-cluster environment.…”

“…Our platform is built as an extension of k8s, KubeFed, and other leading cloud-native tools. Our work has the following objectives: (1) Maximize resource utilization in multi-cluster environments; (2) Guarantee that all applications submitted to a multi-cluster environment find a place to run by making use of all existing resources and provisioning additional resources from the cloud; (3) Maintain the performance of applications by adjusting the number of replicas in response to changing user traffic; and (4) Guarantee that resources are not over-provisioned and wasted unnecessarily when they are not being used. Our experimental results (Section VI) in a geo-distributed multi-cluster environment complemented with resources from the cloud show that mck8s reduces resource fragmentation by balancing resource allocation for multi-cluster applications, and keeping the percentage of pending pods below 6% as compared to 65% in the case of KubeFed for the same workload.…”

Predictive Replica Placement for Mobile Users in Distributed Fog Data Stores with Client-Side Markov Models

“…Replica placement in fog has been discussed both for data [16], and for services [12,7,5]. In a survey by Salaht et al [27] that presents an overview of service placement algorithms in fog computing, the authors note that most current service placement techniques are reactive, i.e., they do not anticipate client movement as we do in this work.…”