The Effect of Process Topology and Load Balancing on Parallel Programming Models for SMP Clusters and Iterative Algorithms

Abstract

This article focuses on the effect of both process topology and load balancing on various programming models for SMP clusters and iterative algorithms. More specifically, we consider nested loop algorithms with constant flow dependencies, that can be parallelized on SMP clusters with the aid of the tiling transformation. We investigate three parallel programming models, namely a popular message passing monolithic parallel implementation, as well as two hybrid ones, that employ both message passing and multi-threading. We conclude that the selection of an appropriate mapping topology for the mesh of processes has a significant effect on the overall performance, and provide an algorithm for the specification of such an efficient topology according to the iteration space and data dependencies of the algorithm. We also propose static load balancing techniques for the computation distribution between threads, that diminish the disadvantage of the master thread assuming all inter-process communication due to limitations often imposed by the message passing library. Both improvements are implemented as compile-time optimizations and are further experimentally evaluated. An overall comparison of the above parallel programming styles on SMP clusters based on micro-kernel experimental evaluation is further provided, as well.

Research of barber model in process concurrency control

“…P and v primitives can be in different process and v operation is known to resume a suspended process with p operation. But if different processes do same work in a same period, it becomes infructuous to solve this kind of complex problems with original experience [8,9] .…”

Uma abordagem de predição da dinâmica comportamental de processos para prover autonomia a ambientes distribuídos

“…Ao longo do tempo, essa abordagem evoluiu de máquinas paralelas (Thelen, 2000;Kuehn & Smith, 1988) para processadores simétricos (SMP) (Hennessy & Patterson, 2002;Drosinos & Koziris, 2006), sistemas multi-core (Balakrishnan et al, 2005;Creeger, 2005) e, recentemente, many-core (Seiler et al, 2008). Além disso, outras técnicas têm sido utilizadas para aumentar o desempenho de elementos de processamento, tais como abordagens Hyper-Threading (Tian et al, 2003) e virtualização de hardware (Smith & Nair, 2005;Aguiar et al, 2008).…”