A proposal for a heterogeneous cluster ScaLAPACK (dense linear solvers)

Beaumont, Olivier; Boudet, Vincent; Petitet, Antoine; Rastello, Fabrice; Robert, Yves

doi:10.1109/12.956091

Cited by 83 publications

(55 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…They report several numerical simulations. As pointed out in the introduction, theoretical results for matrix multiplication and LU decomposition on 2D-grids of heterogeneous processors are reported in [5], while extensions to general 2D partitioning are considered in [6]. See also Lastovetsky and Reddy [31] for another partitioning approach.…”

Section: Inriamentioning

confidence: 99%

“…This corresponds to arranging the n available resources as a (virtual) 2D grid of size p × q (where p.q ≤ n) so that each processor receives a share of the work, i.e., a rectangle, whose area is proportional to its relative computing speed. There are many processor arrangements to consider, and determining the optimal one is a highly combinatorial problem, which has been proven NPcomplete in [5]. In fact, because of the geometric constraints imposed by the 2D processor grid, a perfect load-balancing can only be achieved in some very particular cases.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting Matrix Product on Master-Worker Platforms

Dongarra

Pineau

Robert

et al. 2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

This paper is aimed at designing efficient parallel matrix-product algorithms for heterogeneous master-worker platforms. While matrix-product is well-understood for homogeneous 2D-arrays of processors (e.g., Cannon algorithm and ScaLAPACK outer product algorithm), there are three key hypotheses that render our work original and innovative: -Centralized data. We assume that all matrix files originate from, and must be returned to, the master. The master distributes both data and computations to the workers (while in ScaLAPACK, input and output matrices are initially distributed among participating resources). Typically, our approach is useful in the context of speeding up MATLAB or SCILAB clients running on a server (which acts as the master and initial repository of files).-Heterogeneous star-shaped platforms. We target fully heterogeneous platforms, where computational resources have different computing powers. Also, the workers are connected to the master by links of different capacities. This framework is realistic when deploying the application from the server, which is responsible for enrolling authorized resources.-Limited memory. Because we investigate the parallelization of large problems, we cannot assume that full matrix panels can be stored in the worker memories and re-used for subsequent updates (as in ScaLAPACK). The amount of memory available in each worker is expressed as a given number m i of buffers, where a buffer can store a square block of matrix elements. The size q of these square blocks is chosen so as to harness the power of Level 3 BLAS routines: q = 80 or 100 on most platforms.We have devised efficient algorithms for resource selection (deciding which workers to enroll) and communication ordering (both for input and result messages), and we report a set of numerical experiments on various platforms atÉcole Normale Supérieure de Lyon and Résumé : Ce papier a pour objectif la définition d'algorithmes efficaces pour le produit de matrices en parallèle sur plate-formes maître-esclaves hétérogènes. Bien que le produit de matrices soit bien compris pour des grilles bi-dimensionnelles de processeurs homogènes (cf. l'algorithme de Cannon et le produit externe de ScaLAPACK), trois hypothèses rendent notre travail original: -Données centralisées. Nous supposons que toutes les matrices résident originellement sur le maître, et doivent yêtre renvoyées. Le maître distribue données et calculs aux esclaves (alors que dans ScaLAPACK, les matrices initiales et résultats sont initiallement distribuées aux processeurs participant). Typiquement, notre approche est justifiée dans le contexte de l'accélération de clients MATLAB ou SCILAB s'exécutant sur un serveur (qui se comporte comme le maître et détient initiallement les données).-Plates-formes hétérogènes enétoile. Nous nous intéressonsà des plates-formes complètement hétérogènes dont les ressources de calculs ont des puissances de calcul différentes et dont les esclaves sont reliés au maître par des liens de capacités différentes. Ce cadre d...

show abstract

Section: Inriamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Revisiting Matrix Product on Master-Worker Platforms

Dongarra

Pineau

Robert

et al. 2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

show abstract

“…Beaumont et al [8] discuss data allocation strategies to implement matrix products and dense linear system solvers on heterogeneous computing platforms as a basis for a successful extension of the ScaLAPACK library to heterogeneous platforms. They show that extending the standard ScaLAPACK block-cyclic distribution to heterogeneous 2D grids is difficult.…”

Section: Literature Surveymentioning

confidence: 99%

HeteroMPI+ScaLAPACK: Towards a ScaLAPACK (Dense Linear Solvers) on Heterogeneous Networks of Computers

Reddy¹,

Lastovetsky

2006

High Performance Computing - HiPC 2006

View full text Add to dashboard Cite

Abstract. The paper presents a tool that ports ScaLAPACK programs designed to run on massively parallel processors to Heterogeneous Networks of Computers. The tool converts ScaLAPACK programs to HeteroMPI programs. The resulting HeteroMPI programs do not aim to extract the maximum performance from a Heterogeneous Networks of Computers but provide an easy and simple way to execute the ScaLAPACK programs on such networks with good performance improvements. We demonstrate the efficiency of the resulting HeteroMPI programs by performing experiments with a matrix multiplication application on a local network of heterogeneous computers.

show abstract

“…Thereafter, no further global communication is required. Each scheduler thread uses a 1D load-balancing mechanism [4] to select a requesting thread and an application type.…”

Section: Centralized Lp-based (Lp)mentioning

confidence: 99%