Efficient collective communication in distributed heterogeneous systems

Bhat, Prashanth B.; Raghavendra, C.S.; Prasanna, Viktor K.

doi:10.1016/s0743-7315(03)00008-x

Cited by 82 publications

(80 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We first account for a constant start-up overhead S, paid for initiating the redistribution call. Then we adopt a realistic one-port communication model [18] where a processor can send and receive at most one message at any time-step. Independent communications, involving distinct sender/receiver pairs, can take place in parallel: however, two messages sent by the same processor will be serialized.…”

Section: Fault-free Scenariomentioning

confidence: 99%

Resilient co-scheduling of malleable applications

Benoît

Pottier

Robert

2017

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

Recently, the benefits of co-scheduling several applications have been demonstrated in a fault-free context, both in terms of performance and energy savings. However, large-scale computer systems are confronted by frequent failures, and resilience techniques must be employed for large applications to execute efficiently. Indeed, failures may create severe imbalance between applications, and significantly degrade performance. In this paper, we aim at minimizing the expected completion time of a set of co-scheduled applications. We propose to redistribute the resources assigned to each application upon the occurrence of failures, and upon the completion of some applications, in order to achieve this goal. First, we introduce a formal model and establish complexity results. The problem is NP-complete for malleable applications, even in a faultfree context. Therefore, we design polynomial-time heuristics that perform redistributions and account for processor failures. A fault simulator is used to perform extensive simulations that demonstrate the usefulness of redistribution and the performance of the proposed heuristics.

show abstract

Section: Fault-free Scenariomentioning

confidence: 99%

Resilient co-scheduling of malleable applications

Benoît

Pottier

Robert

2017

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

show abstract

“…a network card of infinite capacity, on each processor. A more realistic model is the one-port model [8,9]. In this model, a given processor can be involved in a single communication at any time-step, either a send or a receive.…”

Section: Communication Contentionmentioning

confidence: 99%

Complexity Results for Throughput and Latency Optimization of Replicated and Data-parallel Workflows

Benoît

Robert

2008

Algorithmica

View full text Add to dashboard Cite

Mapping applications onto parallel platforms is a challenging problem, even for simple application patterns such as pipeline or fork graphs. Several antagonist criteria should be optimized for workflow applications, such as throughput and latency (or a combination). In this paper, we consider a simplified model with no communication cost, and we provide an exhaustive list of complexity results for different problem instances. Pipeline or fork stages can be replicated in order to increase the throughput by sending consecutive data sets onto different processors. In some cases, stages can also be data-parallelized, i.e. the computation of one single data set is shared between several processors. This leads to a decrease of the latency and an increase of the throughput. Some instances of this simple model are shown to be NP-hard, thereby exposing the inherent complexity of the mapping problem. We provide polynomial algorithms for other problem instances. Altogether, we provide solid theoretical foundations for the study of mono-criterion or bi-criteria mapping optimization problems.

show abstract

“…We adopt the sender-receiver model in this paper and improve this bound to 2T + β. Other models for heterogeneous clusters include [6,16].…”

Section: Communication Modelmentioning

confidence: 99%

“…Despite its efficiency in scheduling broadcast in heterogeneous systems, the FNF heuristic does not guarantee optimal broadcast time [2,6] in sender-only model. Since the sender-only model is a special case of the sender-receiver model, FNF is not optimal in the sender-receiver model either.…”

Section: Fnf Not Guarantee Optimal Broadcast Timementioning

confidence: 99%

An Approximation Algorithm for Broadcast Scheduling in Heterogeneous Clusters

Liu

Wang

Guo

2004

Real-Time and Embedded Computing Systems and Applications

View full text Add to dashboard Cite

Abstract. Network of workstation (NOW) is a cost-effective alternative to massively parallel supercomputers. As commercially available off-theshelf processors become cheaper and faster, it is now possible to build a PC or workstation cluster that provides high computing power within a limited budget. However, a cluster may consist of different types of processors and this heterogeneity within a cluster complicates the design of efficient collective communication protocols. This paper shows that a simple heuristic called fastest-node-first (FNF) [2] is very effective in reducing broadcast time for heterogeneous cluster systems. Despite the fact that FNF heuristic does not guarantee an optimal broadcast time for general heterogeneous network of workstation, we prove that FNF always gives near optimal broadcast time in a special case of cluster, and this finding helps us show that FNF delivers guaranteed performance for general clusters. In a previous paper we showed a similar bound on the competitive ratio in a send-only communication model. This paper extends the result to a more realistic sender-receiver model. We show that FNF gives a total broadcast of 2T + β, where T is the optimum time and β is a constant. This improves over the previous bound on 2αT + β [17], where α is a theoretically unbounded ratio of the processor performance in the cluster.

show abstract

Efficient collective communication in distributed heterogeneous systems

Cited by 82 publications

References 18 publications

Resilient co-scheduling of malleable applications

Resilient co-scheduling of malleable applications

Complexity Results for Throughput and Latency Optimization of Replicated and Data-parallel Workflows

An Approximation Algorithm for Broadcast Scheduling in Heterogeneous Clusters

Contact Info

Product

Resources

About