Data‐parallel programming on a network of heterogeneous workstations

Nedeljkovlc, Nenad; Quinn, Michael J.

doi:10.1002/cpe.4330050404

Cited by 28 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Virtual processors are another technique widely used for load balancing in parallel systems and conventional clusters [Nedeljkovic and Quinn 1992;Kale et al 2000]. Each virtual processor is assigned a small portion of the entire workload and the system dynamically maps these virtual processors to physical processors to achieve load balance.…”

Section: Related Workmentioning

confidence: 99%

Tunable randomization for load management in shared-disk clusters

Burns

2005

ACM Trans. Storage

View full text Add to dashboard Cite

We develop and evaluate a system for load management in shared-disk file systems built on clusters of heterogeneous computers. It balances workload by moving file sets among cluster server nodes. It responds to changing server resources that arise from failure and recovery, and dynamically adding or removing servers. It also realizes performance consistency-nearly uniform performance across all servers. The system is adaptive and self-tuning. It operates without any a priori knowledge of workload properties, or the capabilities of the servers. Rather, it continuously tunes load placement using a technique called adaptive, nonuniform (ANU) randomization. ANU randomization realizes the scalability and metadata reduction benefits of hash-based, randomized placement techniques, while avoiding hashing's drawbacks: load skew, inability to cope with heterogeneity, and lack of tunability. ANU randomization outperforms virtual-processor approaches to load balancing, while reducing the amount of shared state among servers and the amount of load movement.

show abstract

Section: Related Workmentioning

confidence: 99%

Tunable randomization for load management in shared-disk clusters

Burns

2005

ACM Trans. Storage

View full text Add to dashboard Cite

show abstract

“…balancing on a workstation network is to predict future performance based on past information. For example, in [16], a global distributed scheme is presented, and load balancing involves periodic information exchanges. Dome [17] implements a global central and a local distributed scheme, and the load balancing involves periodic exchanges.…”

Section: Predicting the Future A Common Approach Taken For Loadmentioning

confidence: 99%

Compile-Time Scheduling Algorithms for a Heterogeneous Network of Workstations

Cierniak

Zaki

1997

The Computer Journal

View full text Add to dashboard Cite

In this paper, we study the problem of scheduling parallel loops at compile time for a heterogeneous network of workstations. We consider heterogeneity in various aspects of parallel programming: program, processor, memory and network. A heterogeneous program has parallel loops with different amounts of work being done in each iteration; heterogeneous processors have different speeds; heterogeneous memory refers to the different amounts of user-available memory on the machines and a heterogeneous network has different communication costs between processors. We propose a simple yet comprehensive model for use in compiling for a network of processors, and develop compiler algorithms for generating optimal and near-optimal schedules of loops for load balancing, communication optimizations, network contention and memory heterogeneity. Experiments show that a signi"cant performance improvement is achieved using our techniques.

show abstract

“…For example, the PDU might be a row, column, or block of a matrix in a matrix-based problem, or a collection of particles in a particle simulation. The PDU is similar to the virtual processor [5,6] and may arise from unstructured data domains. The partitioning method does not depend on the nature of the PDU but rather manipulates PDUs in the abstract.…”

Section: Data Parallel Computationsmentioning

confidence: 99%

“…For each configuration explored, we compute T, via (9). To do this we first compute the partitionmap A, via (6). Once A, is determined, we can compute Tcomp and Tcomnl easily by invoking the callbacks and selecting the appropriate communication function.…”

Section: Processor Selectionmentioning

confidence: 99%

“…The programmer is responsible for problem decomposition and selecting the number of processors. Dataparallel C [5,6] provides a language and run-time system that supports both static and dynamic partitioning of regular data parallel computations across heterogeneous workstation network:;. Dataparallel C is limited to workstations or multicomputers (but not both) and does not provide granularity control.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A framework for partitioning parallel computations in heterogeneous environments

Weissman

Grimshaw

1995

Concurrency: Pract. Exper.

View full text Add to dashboard Cite

In the paper we present a framework for partitioning data parallel computations across a heterogeneous metasystem at runtime. The framework is guided by program and resource information which is made available to the system. Three difficult problems are handled by the framework: processor selection, task placement and heterogeneous data domain decomposition. Solving each of these problems contributes to reduced elapsed time. In particular, processor selection determines the best gain size at which to run the computation, task placement reduces communication cost, and data domain decomposition achieves processor load balance. We present results which indicate that excellent performanceis achievableusing the framework. The paper extends our earlier work on partitioning data parallel computations across a singlelevel network of heterogeneous workstations. INTRODUCTIONA great deal of recent interest has been sparked within academic, industrial and government circles in the emerging technology of metmystem-based high-performance computing. A metasystern is a shared ensemble of workstations, vector and parallel machines connected by local-and wide-area networks (see Figure I). The promise of on-line gigabit networks coupled with the tremendous computing power of the metasystem makes it very attractive for parallel computations.The potentially large array of heterogeneous resources in the metasystem offers an opportunity for delivering high performance on a range of parallel computations. Choosing the best set of available resources is a difficult problem and is the subject of this paper. Consider the set of machines in Table 1 and observe that they have different computation and communication capacities. Loosely coupled parallel computations with infrequent communication would probably benefit by applying the fastest set of computational resources (perhaps the DEC-Alphacluster), and may benefit from distributionacross many machines. On the other hand, more tightly coupled parallel computations are best suited to machines that have a higher communication capacity (perhaps an Intel Paragon), but may also benefit from distribution across many machines if the computation granularity is sufficient. We address the latter problem in this paper.We present a framework that automates partitioning and placement of data parallel computations across metasystems such as in Figure 1. Partitioning is performed at runtime when the state of the metasystem resources are known. Three difficult problems are handled by the framework: processor selection, task placement and heterogeneous data domain decomposition. Solving each of these problems contributes to reduced completion time. Processor selection chooses the best number and type of processors to apply to the computation. This el = k = T = the i " ' network cluster the irh processor cluster application communication topology message size in bytes communication cost coefficients processor-dependent communication function router cost constants coercion cost constant number of messages th...

show abstract

Data‐parallel programming on a network of heterogeneous workstations

Cited by 28 publications

References 10 publications

Tunable randomization for load management in shared-disk clusters

Tunable randomization for load management in shared-disk clusters

Compile-Time Scheduling Algorithms for a Heterogeneous Network of Workstations

A framework for partitioning parallel computations in heterogeneous environments

Contact Info

Product

Resources

About