Parallel MATLAB: Doing it Right

Abstract

MATLAB is one of the most widely used mathematical computing environments in technical computing. It is an interactive environment that provides high-performance computational routines and an easy-to-use, C-like scripting language. It started out as an in-1) Memory model: Distributed memory was the dominant model for parallel computers, and for linear algebra applications, scatter/gather of the matrix took too long to make parallel computation worthwhile. 2) Granularity: For typical use, MATLAB spends most of its time in the parser, interpreter and graphics routines, where any parallelism is difficult to find. Also, to handle embarrassingly parallel applications, which only requires a collection of results at the end, MATLAB would require fundamental changes in its architecture. 3) Business situation: There were not enough customers with parallel computers to support the development. It has been nine years since the article was written, and we have seen tremendous changes in the computing world. These changes have invalidated the arguments that there should not be a parallel MATLAB. 1) Memory model: As modern scientific and engineering problems grow in complexity, the computation time and memory requirements skyrocket. The increase in processor speed and the amount of memory that can fit in a single machine could not catch up with the pace of computation requirements. Very often, current scientific problems simply do not fit into the memory of a single machine, making parallel computation a necessity. This has always been true throughout the history of computing-problems just do not fit into memory. But we have hit the point where a lot of interesting and practical problems from different areas do not fit into memory of a single machine. This has made parallel computing a necessity.

Auto-tuning a high-level language targeted to GPU codes

“…There are a number of library generators that automatically produce high-performance kernels, including FFT [13,29,36], BLAS [34,37,5,14,16], Sparse Numerical Computation [19,33,26,4,23], and domain specific routines [3,7,24]. Recent research [21,22,17] expands automatic code generation to routines whose performance depends not only on architectural features, but also on input characteristics.…”

Lean and Efficient Business Tier for Performance Scaling

“…The business processing is supposed to deliver high throughput, fast response time along with accuracy, consistency and reliability. The challenge is that the business tier needs to cope up with high degree of concurrency issues arising out of handling state-full data as against the other tiers where majority of the handled data is stateless and naturally paves way for high level of concurrency and parallelism otherwise termed as Embarrassingly Parallel [7].…”

Loop Parallelisation for the Jikes RVM

“…This work has included automatic parallelisation, that has no need for programmer intervention or compiler constructs that imply dependencies can be ignored. GCC 4 fits this criterion, as do Intel's Fortran/C compiler and Matlab*P [12,2,6].…”