Automatic Parallelization of Scripting Languages: Toward Transparent Desktop Parallel Computing

Abstract

Desktop computing remains indispensable in scientific exploration, largely because it provides people with devices for human interaction and environments for interactive job execution. However, with today's rapidly growing data volume and task complexity, it is increasingly hard for individual workstations to meet the demands of interactive scientific data processing. The increasing cost of such interactive processing is hindering the productivity of end-to-end scientific computing workflows. While existing distributed computing systems allow people to aggregate desktop workstation resources for parallel computing, the burden of explicit parallel programming and parallel job execution often prohibits scientists to take advantage of such platforms. In this paper, we discuss the need for transparent desktop parallel computing in scientific data processing. As an initial step toward this goal, we present our ongoing work on the automatic parallelization of the scripting language R, a popular tool for statistical computing. Our preliminary results suggest that a reasonable speedup can be achieved on real-world sequential R programs without requiring any code modification. 1 1 This research was funded by an NSF CAREER Award, CNS-0546301 and Xiaosong Ma's joint appointment with Oak Ridge National Laboratory (ORNL). It was also partly supported by the DOE Scientific Data Management Center (http://sdmcenter.lbl.gov). ORNL is managed by the University of Tennessee-Battelle, L.L.C. for the Department of Energy under contract DOE-AC05-00OR22725.

A Survey on Parallel Computing and its Applications in Data-Parallel Problems Using GPU Architectures

“…Today, pH (parallel Haskell) is probably the first fully implicit parallel programming language [91]. Automatic parallelization is hard to achieve for algorithms not based on simple loops and has become a research topic in the last twenty years [48,74,79,105].…”

High Performance Data Mining Using R on Heterogeneous Platforms

“…Instead, they are stored in a server, and accessed through network sockets, thus enabling a threads-like view. Parallel-R [7] and pR [8] enable the statistical analysis routines available in R to be deployed on high performance architecture.…”

Machine learning based online performance prediction for runtime parallelization and task scheduling

“…Although we propose a general purpose approach, we present our new Adaptively Scheduled parallel R (ASpR) framework within the context of our test platform, the pR framework [8] for transparent R script execution [9]. pR takes a sequential R script as input and transparently executes it in parallel using a master-worker model.…”

“…However, automatic parallelization remains an elusive goal, despite much work to parallelize general-purpose programming languages transparently [4], [5], [6]. Alternatively, automatic parallelization of scripting languages [7], [8] would alleviate the application developers' parallel programming burden while avoiding many of the difficulties of parallelizing compilers. This approach, which covers not only scientific applications but also those traditionally run on commercial servers, personal computers or even cell phones, must transparently exploit both task parallelism and data parallelism.…”

“…We use artificial neural networks (ANNs), which are application-independent and self-learning black box models, as required for implicit parallelization. We integrate these techniques into the existing pR framework [8], which performs runtime, automatic parallelization of R scripts. The result is an adaptive scheduling framework for the parallel execution of R, which we call ASpR (Adaptively Scheduled parallel R -pronounced "aspire").…”