Transformations to Parallel Codes for Communication-Computation Overlap

Danalis, Anthony; Kim, Ki-Yong; Pollock, Lori; Swany, Martin

doi:10.1109/sc.2005.75

Cited by 58 publications

(40 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [21] the overlapping schedule for the class of applications under consideration was proposed, while in [13] this schedule was coupled with an underlying SCI network [27] to implement actual overlapping. Danalis et al [14] present a method to transform MPI programs directed towards improving communication-computation overlap in MPI collective operations. The approach is verified using Myrinet.…”

Section: Related Workmentioning

confidence: 99%

Overlapping computation and communication in SMT clusters with commodity interconnects

Goumas

Anastopoulos

Koziris

et al. 2009

2009 IEEE International Conference on Cluster Computing and Workshops

View full text Add to dashboard Cite

Abstract-In this paper we focus on optimizing the performance in a cluster of Simultaneous Multithreading (SMT) processors connected with a commodity interconnect (e.g. Gbit Ethernet), by applying overlapping of computation with communication. As a test case we consider the parallelized advection equation and discuss the steps that need to be followed to semantically allow overlapping to occur. We propose an implementation based on the concept of Helper Threading that distributes computation and communication in the two sibling threads of an SMT processor, thus creating an asymmetric pair of execution patterns in each hardware context. Our experimental results in an 8-node cluster interconnected with commodity Gbit Ethernet demonstrate that the proposed implementation is able to achieve substantial performance improvements that can exceed 20% in some cases, by efficiently utilizing the available resources of the SMT processors.

show abstract

Section: Related Workmentioning

confidence: 99%

Overlapping computation and communication in SMT clusters with commodity interconnects

Goumas

Anastopoulos

Koziris

et al. 2009

2009 IEEE International Conference on Cluster Computing and Workshops

View full text Add to dashboard Cite

show abstract

“…Brightwell et al [7] classifies the source of performance advantage for overlap and Dimitrov [8] uses overlapping as fundamental approach to optimize parallel applications for cluster systems. Other studies, as [9,10,11,12] apply several transformations to parallel codes to enable overlapping. However, little research has been done in the field of non-blocking collectives.…”

Section: Related Workmentioning

confidence: 99%

A Case for Non-blocking Collective Operations

Hoefler

Squyres

Rehm

et al. 2006

Frontiers of High Performance Computing and Networking – ISPA 2006 Workshops

View full text Add to dashboard Cite

Abstract. Non-blocking collective operations for MPI have been in discussion for a long time. We want to contribute to this discussion and to give a rationale for the usage these operations and assess their possible benefits. A LogGP model for the CPU overhead of collective algorithms and a benchmark to measures it are provided and show a large potential to overlap communication and computation. We show that nonblocking collective operations can provide at least the same benefits as non-blocking point to point operations already do. Our claim is that actual CPU overhead for non-blocking collective operations depends on the message size and the communicator size and benefits especially highly scalable applications with huge communicators. We prove that the share of the overhead of the overall communication time of current blocking collective operations gets smaller with bigger communicators and larger messages. We show that the user level CPU overhead is less than 10% for MPICH2 and LAM/MPI using TCP/IP communication, which leads us to the conclusion that, by using non-blocking collective communication, ideally 90% idle CPU time can be freed for the application.

show abstract

“…We can classify previous works into three kinds. Some of researches are achieved by compiled methods [1,2,3,4,5]; some of them have been performed in the field of Global Address Space languages [6,7,8] or achieved by particular hardware [9,10,11]. However, these techniques may be effective for overlapping communication and computation only in a single loop.…”

Section: Introductionmentioning

confidence: 99%

Automatic Transformation for Overlapping Communication and Computation

Shao

Wang

et al. 2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Message-passing is a predominant programming paradigm for distributed memory systems. RDMA networks like infiniBand and Myrinet reduce communication overhead by overlapping communication with computation. For the overlap to be more effective, we propose a source-to-source transformation scheme by automatically restructuring message-passing codes. The extensions to control-flow graph can accurately analyze the message-passing program and help perform data-flow analysis effectively. This analysis identifies the minimal region between producer and consumer, which contains message-passing functional calls. Using inter-procedural data-flow analysis, the transformation scheme enables the overlap of communication with computation. Experiments on the well-known NAS Parallel Benchmarks show that for distributed memory systems, versions employing communication-computation overlap are faster than original programs.

show abstract

Transformations to Parallel Codes for Communication-Computation Overlap

Cited by 58 publications

References 22 publications

Overlapping computation and communication in SMT clusters with commodity interconnects

Overlapping computation and communication in SMT clusters with commodity interconnects

A Case for Non-blocking Collective Operations

Automatic Transformation for Overlapping Communication and Computation

Contact Info

Product

Resources

About