SPRINT: A new parallel framework for R

Abstract

BackgroundMicroarray analysis allows the simultaneous measurement of thousands to millions of genes or sequences across tens to thousands of different samples. The analysis of the resulting data tests the limits of existing bioinformatics computing infrastructure. A solution to this issue is to use High Performance Computing (HPC) systems, which contain many processors and more memory than desktop computer systems. Many biostatisticians use R to process the data gleaned from microarray analysis and there is even a dedicated group of packages, Bioconductor, for this purpose. However, to exploit HPC systems, R must be able to utilise the multiple processors available on these systems. There are existing modules that enable R to use multiple processors, but these are either difficult to use for the HPC novice or cannot be used to solve certain classes of problems. A method of exploiting HPC systems, using R, but without recourse to mastering parallel programming paradigms is therefore necessary to analyse genomic data to its fullest.ResultsWe have designed and built a prototype framework that allows the addition of parallelised functions to R to enable the easy exploitation of HPC systems. The Simple Parallel R INTerface (SPRINT) is a wrapper around such parallelised functions. Their use requires very little modification to existing sequential R scripts and no expertise in parallel computing. As an example we created a function that carries out the computation of a pairwise calculated correlation matrix. This performs well with SPRINT. When executed using SPRINT on an HPC resource of eight processors this computation reduces by more than three times the time R takes to complete it on one processor.ConclusionSPRINT allows the biostatistician to concentrate on the research problems rather than the computation, while still allowing exploitation of HPC systems. It is easy to use and with further development will become more useful as more functions are added to the framework.

SCBI_MapReduce, a New Ruby Task-Farm Skeleton for Automated Parallelisation and Distribution in Chunks of Sequences: The Implementation of a Boosted Blast+

“…Parallelisation libraries for R language, besides Rmpi, are SPRINT [24] and pR [25] packages, whose their main advantage is that they require very little modification to the existing sequential R scripts and no expertise in parallel computing; however, the master worker suffers from communication overhead, and the authors recognise that their approach may not yield the optimal schedule [25]. Other parallelisation libraries are snow and nws that provide coordination and parallel execution facilities.…”

Parallel classification and feature selection in microarray data using SPRINT

“…Adding a new function involves writing the parallel algorithm (typically in C or Fortran) and an interface function in R [7]. This approach makes for much better ease of use by bioinformaticians: in most cases, their existing analysis scripts will need only minor changes.…”

“…The SPRINT [7] project aims to implement a library that biostatisticians can use to exploit HPC systems while requiring only minimal changes to their existing analysis workflows. SPRINT provides drop-in replacements for a number of computationally expensive R functions that were identified as important in a user requirements survey of the bioinformatics community [8].…”

A distributed computing framework for All-to-All comparison problems

“…Parallel All-to-All Comparisons of genome sequences are considered by Hill et al [8]. The main focus of this work is on load balancing among the clusters by dividing the comparison matrix into rows and assigning these rows to different nodes dynamically.…”

“…There have been several efforts to address All-to-All Comparison Problems [4,5,6,7,8,9]. A main approach is to provide solutions for special-purpose All-to-All Comparison Problems only.…”