The multi-commodity flow-cut gap is a fundamental parameter that affects the performance of several divide & conquer algorithms, and has been extensively studied for various classes of undirected graphs. It has been shown by Linial, London and Rabinovich [15] and by Aumann and Rabani [3] that for general n-vertex graphs it is bounded by Oplog nq and the Gupta-Newman-Rabinovich-Sinclair conjecture [9] asserts that it is Op1q for any family of graphs that excludes some fixed minor.The flow-cut gap is poorly understood for the case of directed graphs. We show that for uniform demands it is Op1q on directed series-parallel graphs, and on directed graphs of bounded pathwidth. These are the first constant upper bounds of this type for some non-trivial family of directed graphs. We also obtain Op1q upper bounds for the general multi-commodity flow-cut gap on directed trees and cycles. These bounds are obtained via new embeddings and Lipschitz quasipartitions for quasimetric spaces, which generalize analogous results form the metric case, and could be of independent interest. Finally, we discuss limitations of methods that were developed for undirected graphs, such as random partitions, and random embeddings. has a non-negative demand dempiq. A cut in G is a subset of directed edges of EpGq. For a cut S Ď EpGq in G, let I S be the set of all indices i P t1, 2, . . . , ku such that all paths from s i to t i have at least one edge in S. Let DpSq " ř iPI S dempiq be the demand separated by S. Let W pSq " CpSq DpSq be the sparsity of S. The goal is to find the cut with minimum sparsity. The LP relaxation of this problem corresponds to the dual of the LP formulation of the directed maximum concurrent flow problem, and the integrality gap of this LP relaxation is the directed multi-commodity flow-cut gap. Hajiyaghayi and Räcke [10] showed an upper bound of Op ? nq for the flow-cut gap. This upper bound on the gap has been further improved by Agarwal, Alon and Charikar toÕpn 11{23 q in [1]. For directed graphs of treewidth t, it has been shown that the gap is at most t log Op1q n by Mémoli, Sidiropoulos and Sridhar [16]. On the lower bound side Saks et al. [17] showed that for general directed graphs the flow-cut gap is at least k´ε, for any constant ε ą 0, and for any k " Oplog n{ log log nq. Chuzhoy and Khanna showed aΩpn
Parallel aggregation is a ubiquitous operation in data analytics that is expressed as GROUP BY in SQL, reduce in Hadoop, or segment in TensorFlow. Parallel aggregation starts with an optional local pre-aggregation step and then repartitions the intermediate result across the network. While local pre-aggregation works well for low-cardinality aggregations, the network communication cost remains significant for high-cardinality aggregations even after local pre-aggregation. The problem is that the repartition-based algorithm for high-cardinality aggregation does not fully utilize the network. In this work, we first formulate a mathematical model that captures the performance of parallel aggregation. We prove that finding optimal aggregation plans from a known data distribution is NP-hard, assuming the Small Set Expansion conjecture. We propose GRASP, a GReedy Aggregation Scheduling Protocol that decomposes parallel aggregation into phases. GRASP is distribution-aware as it aggregates the most similar partitions in each phase to reduce the transmitted data size in subsequent phases. In addition, GRASP takes the available network bandwidth into account when scheduling aggregations in each phase to maximize network utilization. The experimental evaluation on real data shows that GRASP outperforms repartition-based aggregation by 3.5x and LOOM by 2.0x.
Parallel aggregation is a ubiquitous operation in data analytics that is expressed as GROUP BY in SQL, reduce in Hadoop, or segment in TensorFlow. Parallel aggregation starts with an optional local pre-aggregation step and then repartitions the intermediate result across the network. While local preaggregation works well for low-cardinality aggregations, the network communication cost remains significant for highcardinality aggregations even after local pre-aggregation. The problem is that the repartition-based algorithm for highcardinality aggregation does not fully utilize the network.In this work, we first formulate a mathematical model that captures the performance of parallel aggregation. We prove that finding optimal aggregation plans from a known data distribution is NP-hard, assuming the Small Set Expansion conjecture. We propose GRASP, a GReedy Aggregation Scheduling Protocol that decomposes parallel aggregation into phases. GRASP is distribution-aware as it aggregates the most similar partitions in each phase to reduce the transmitted data size in subsequent phases. In addition, GRASP takes the available network bandwidth into account when scheduling aggregations in each phase to maximize network utilization. The experimental evaluation on real data shows that GRASP outperforms repartition-based aggregation by 3.5× and LOOM by 2.0×.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.