Identifying the capability of overlapping computation with communication

Sohn, Andrew; Ku, Jui-Yuan; Kodama, Y.; Sato, Mitsuhisa; Sakane, Hitoshi; Yamana, Hayato; Sakai, Shuichi; Yamaguchi, Yoshihiro

doi:10.1109/pact.1996.552657

Cited by 14 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sohn et al [15] tested various multiprocessors and compared their overlapping efficiency. Later work studied many MPI implementations and showed that their overlapping abilities are different [4].…”

Section: Related Workmentioning

confidence: 99%

The Impact of Application's Micro-Imbalance on the Communication-Computation Overlap

Subotic

Sancho

Labarta

et al. 2011

2011 19th International Euromicro Conference on Parallel, Distributed and Network-Based Processing

View full text Add to dashboard Cite

Although the community sees overlapping communication and computation as a perspective avenue for advancing parallel execution, it remains unclear what type of applications, under which conditions, and to which extent could benefit from this technique. To tackle this issue, we designed a simulation environment that allowed us to profoundly study overlap. We found out that overlapping potential in an application is determined by the application's parallel behavior and the pattern by which each process locally produces/consumes data involved in communication. We identified two behaviors that directly influence the application's overlapping potential -we name them microscopic imbalance of computation and microscopic imbalance of communication.In an application that expresses some of these two behaviors, a fine-grain overlapping technique can achieve a significant execution speedup, a speedup that can even be higher than 2. We believe that our findings can help a programmer estimate how much his application could benefit from overlap, and therefore decide whether implementing that technique is worth the effort.

show abstract

Section: Related Workmentioning

confidence: 99%

The Impact of Application's Micro-Imbalance on the Communication-Computation Overlap

Subotic

Sancho

Labarta

et al. 2011

2011 19th International Euromicro Conference on Parallel, Distributed and Network-Based Processing

View full text Add to dashboard Cite

show abstract

“…Sohn et al [24] tested various multiprocessors and compared their overlapping efficiency. Furthermore, Brightwell et al [5] quantified in detail how performance is influenced by the network properties of overlap, offload and independent progress.…”

Section: Related Workmentioning

confidence: 99%

A Simulation Framework to Automatically Analyze the Communication-Computation Overlap in Scientific Applications

Subotic

Sancho

Labarta

et al. 2010

2010 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

“…The paper showed that latency hiding technique increases the feasibility of parallel computing in high latency networks of workstations across the Internet as well as in multiprocessor systems. In [9], the effect of hiding latency by overlapping computation with communication was investigated for runtime-dependent network contention.…”

Section: Related Workmentioning

confidence: 99%

“…To fully employ the computational power of the networked computing devices, latency hiding by overlapping computation with communication is a well known technique [9,10]. It can be achieved by partitioning the input data into several data pieces, then pipelining the data pieces into the network.…”

Section: Introductionmentioning

confidence: 99%