Towards Distributed Lexicographically Fair Resource Allocation with an Indivisible Constraint

Abstract

In the cloud computing and big data era, data analysis jobs are usually executed over geo-distributed data centers to make use of data locality. When there are not enough resources to fully meet the demands of all the jobs, allocating resources fairly becomes critical. Meanwhile, it is worth noting that in many practical scenarios, resources waiting to be allocated are not infinitely divisible. In this paper, we focus on fair resource allocation for distributed job execution over multiple sites, where resources allocated each time have a minimum requirement. Aiming at the problem, we propose a novel scheme named Distributed Lexicographical Fairness (DLF) targeting to well specify the meaning of fairness in the new scenario considered. To well study DLF, we follow a common research approach that first analyzes its economic properties and then proposes algorithms to output concrete DLF allocations. In our study, we leverage a creative idea that transforms DLF equivalently to a special max flow problem in the integral field. The transformation facilitates our study in that by generalizing basic properties of DLF from the view of network flow, we prove that DLF satisfies Pareto efficiency, envy-freeness, strategy-proofness, relaxed sharing incentive and 12-maximin share. After that, we propose two algorithms. One is a basic algorithm that stimulates a water-filling process. However, our analysis shows that the time complexity is not strongly polynomial. Aiming at such inefficiency, we then propose a new iterative algorithm that comprehensively leverages parametric flow and push-relabel maximal flow techniques. By analyzing the steps of the iterative algorithm, we show that the time complexity is strongly polynomial.

PRISM based Transport

“…The LMMF vector doesn't consist of just the max min rate (arg max x 0 ) but also the maximums of any other rate (x i ) given the values of the lower rates (x 0 , ..., x i−1 ). Therefore, LMMF achieves Pareto efficiency [28], [30], [33] and always exists (as long as the number of flows is finite) [34].…”