An Evaluation of One-Sided and Two-Sided Communication Paradigms on Relaxed-Ordering Interconnect

Ibrahim, Khaled Z.; Hargrove, Paul; Iancu, Costin; Yelick, Katherine

doi:10.1109/ipdps.2014.116

Cited by 10 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performance advantage of UPC's one-sided communication over the MPI counterpart has also been reported in e.g. [16]. On the other hand, persistent advantages of MPI over UPC include better data locality and more flexible data partitionings.…”

Section: Discussionmentioning

confidence: 64%

Performance Optimization and Modeling of Fine-Grained Irregular Communication in UPC

Lagravière

Langguth

Prugger

et al. 2019

Scientific Programming

View full text Add to dashboard Cite

The UPC programming language offers parallelism via logically partitioned shared memory, which typically spans physically disjoint memory sub-systems. One convenient feature of UPC is its ability to automatically execute between-thread data movement, such that the entire content of a shared data array appears to be freely accessible by all the threads. The programmer friendliness, however, can come at the cost of substantial performance penalties. This is especially true when indirectly indexing the elements of a shared array, for which the induced between-thread data communication can be irregular and have a fine-grained pattern. In this paper we study performance enhancement strategies specifically targeting such fine-grained irregular communication in UPC. Starting from explicit thread privatization, continuing with block-wise communication, and arriving at message condensing and consolidation, we obtained considerable performance improvement of UPC programs that originally require fine-grained irregular communication. Besides the performance enhancement strategies, the main contribution of the present paper is to propose performance models for the different scenarios, in form of quantifiable formulas that hinge on the actual volumes of various data movements plus a small number of easily obtainable hardware characteristic parameters. These performance models help to verify the enhancements obtained, while also providing insightful predictions of similar parallel implementations, not limited to UPC, that also involve between-thread or between-process irregular communication. As a further validation, we also apply our performance modeling methodology and hardware characteristic parameters to an existing UPC code for solving a 2D heat equation on a uniform mesh. MotivationGood programmer productivity and high computational performance are usually two conflicting goals in the context of developing parallel code for scientific computations. Partitioned global address space (PGAS) [10,2,11,22], however, is a parallel programming model that aims to achieve both goals at the same time. The fundamental mechanism of PGAS is a global address space that is conceptually shared among concurrent processes that jointly execute a parallel program. Data exchange between the processes is carried out by a low-level network layer "under the hood" without explicit involvement from the programmer, thus providing good productivity. The shared global address space is logically partitioned such that each partition has affinity to a designated owner process. This awareness of data locality is essential for achieving good performance of parallel programs written in the PGAS model, because the globally shared address space may actually encompass many physically distributed memory sub-systems.Unified Parallel C (UPC) [13,28] is an extension of the C language and provides the PGAS parallel programming model. The concurrent execution processes of UPC are termed as threads, which execute a UPC program in the style of single-program-multiple-...

show abstract

Section: Discussionmentioning

confidence: 64%

Performance Optimization and Modeling of Fine-Grained Irregular Communication in UPC

Lagravière

Langguth

Prugger

et al. 2019

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…HyperTransport allows lower injection latency for small transfers compared with the newer generation Cray Aries interconnect. Tracking remote completion for put operations on Gemini is more complex especially with relaxed ordering of transfers (Ibrahim et al 2014), making puts slower than the get operations by up to 20%.…”

Section: Network Performance and Saturationmentioning

confidence: 99%

Reaching bandwidth saturation using transparent injection parallelization

Chaimov

Ibrahim

Williams

et al. 2016

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

Although logically available, applications may not exploit enough instantaneous communication concurrency to maximize network utilization on HPC systems. This is exacerbated in hybrid programming models that combine single program multiple data with OpenMP or CUDA. We present the design of a “multi-threaded” runtime able to transparently increase the instantaneous network concurrency and to provide near saturation bandwidth, independent of the application configuration and dynamic behavior. The runtime offloads communication requests from application level tasks to multiple communication servers. The servers use system specific performance models to attain network saturation. Our techniques alleviate the need for spatial and temporal application level message concurrency optimizations. Experimental results show improved message throughput and bandwidth by as much as 150% for 4 KB messages on InfiniBand and by as much as 120% for 4 KB messages on Cray Aries. For more complex operations such as all-to-all collectives, we observe as much as 30% speedup. This translates into 23% speedup on 12,288 cores for a NAS FT implemented using FFTW. We observe as much as 76% speedup on 1500 cores for an already optimized UPC+OpenMP geometric multigrid application using hybrid parallelism. For the geometric multigrid GPU implementation, we observe as much as 44% speedup on 512 GPUs.

show abstract

“…MPI is implemented as library since it enables convenient program that can manage to run similar program on parallel processors and has gone through enhancements in various versions [ 25 – 27 ]. Communication using MPI can be point-to-point or it can be collective communication [ 28 ]. The basic methodology of MPI is having multiple processes on distributed memory systems which communicate using message passing.…”

Section: Introductionmentioning

confidence: 99%

Optimizing NEURON Simulation Environment Using Remote Memory Access with Recursive Doubling on Distributed Memory Systems

Shehzad

Bozkuź

2016

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Increase in complexity of neuronal network models escalated the efforts to make NEURON simulation environment efficient. The computational neuroscientists divided the equations into subnets amongst multiple processors for achieving better hardware performance. On parallel machines for neuronal networks, interprocessor spikes exchange consumes large section of overall simulation time. In NEURON for communication between processors Message Passing Interface (MPI) is used. MPI_Allgather collective is exercised for spikes exchange after each interval across distributed memory systems. The increase in number of processors though results in achieving concurrency and better performance but it inversely affects MPI_Allgather which increases communication time between processors. This necessitates improving communication methodology to decrease the spikes exchange time over distributed memory systems. This work has improved MPI_Allgather method using Remote Memory Access (RMA) by moving two-sided communication to one-sided communication, and use of recursive doubling mechanism facilitates achieving efficient communication between the processors in precise steps. This approach enhanced communication concurrency and has improved overall runtime making NEURON more efficient for simulation of large neuronal network models.

show abstract

An Evaluation of One-Sided and Two-Sided Communication Paradigms on Relaxed-Ordering Interconnect

Cited by 10 publications

References 19 publications

Performance Optimization and Modeling of Fine-Grained Irregular Communication in UPC

Performance Optimization and Modeling of Fine-Grained Irregular Communication in UPC

Reaching bandwidth saturation using transparent injection parallelization

Optimizing NEURON Simulation Environment Using Remote Memory Access with Recursive Doubling on Distributed Memory Systems

Contact Info

Product

Resources

About