Distributed Parallel Computing Using Navigational Programming

Abstract

Message Passing (MP) and Distributed Shared Memory (DSM) are the two most common approaches to distributed parallel computing. MP is difficult to use, whereas DSM is not scalable. Performance scalability and ease of programming can be achieved at the same time by using navigational programming (NavP). This approach combines the advantages of MP and DSM, and it balances convenience and flexibility. Similar to MP, NavP suggests to its programmers the principle of pivot-computes and hence is efficient and scalable. Like DSM, NavP supports incremental parallelization and shared variable programming and is therefore easy to use. The implementation and performance analysis of real-world algorithms, namely parallel Jacobi iteration and parallel Cholesky factorization, presented in this paper supports the claim that the NavP approach is better suited for general-purpose parallel distributed programming than either MP or DSM.

Mobile Pipelines: Parallelizing Left-Looking Algorithms Using Navigational Programming

“…(We note that when data parallelism is present, we can use multiple concurrent DSC threads to exploit this data parallelism [6], but this is not the focus of this paper.) (3) The NavP code as listed in Fig.…”

“…= 1) (6) recv cnt ← recv cnt + 1 (7) if (rank ! = node map [1] Call request proc(i, j, x) (57) end while Figure 11(d) lists the pseudocode for a pipelined DPC implementation.…”

“…We are unable to find a parallel Crout implementation in literature with reasonable search efforts. For performance comparison we compare our speedup numbers with those of the Cholesky factorization implementation in ScaLA-PACK [11] and find that they are comparable [6]. We are comparing the performances of two different algorithms based on the observation that they are two variants of LU decomposition and have asymptotically the same complexity.…”

“…The array a[. ] is distributed as follows: a [6] and a [2] are on node n1, a [1] and a [4] are on node n2, and a [3] and a [5] are on node n3. This distribution is shown in Fig.…”

Toward Incremental Parallelization Using Navigational Programming

“…Navigational Programming (NavP) is a methodology for distributed parallel programming based on the use of selfmigrating computations [1]. In NavP code, a programmer inserts navigational commands, i.e., hop() statements, to migrate computation locus in order to access remotely distributed data and spread out computations.…”