State of the Art in Parallel Computing with<i>R</i>

Schmidberger, Markus; Morgan, Martin; Eddelbuettel, Dirk; Yu, Hao; Tierney, Luke; Mansmann, Ulrich

doi:10.18637/jss.v031.i01

Cited by 106 publications

(82 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two main architectural approaches are present for the processing of large amounts of statistical data: scaling the statistical tool or scaling the data management system [39]. Systems in the first category tend to stick to a specific tool and language and provide a scalable infrastructure for it.…”

Section: Scaling In Statistical Data Processingmentioning

confidence: 99%

Executable schema mappings for statistical data processing

Atzeni

Bellomarini

Bugiotti

et al. 2017

Distrib Parallel Databases

View full text Add to dashboard Cite

Data processing is the core of any statistical information system. Statisticians are interested in specifying transformations and manipulations of data at a high level, in terms of entities of statistical models. We illustrate here a proposal where a high-level language, EXL, is used for the declarative specification of statistical programs, and a translation into executable form in various target systems is available. The language is based on the theory of schema mappings, in particular those defined by a specific class of tgds, which we actually use to optimize user programs and facilitate the translation towards several target systems. The characteristics of such class guarantee good tractability properties and the applicability in Big Data settings. A concrete implementation, EXLEngine, has been carried out and is currently used at the Bank of Italy.

show abstract

Section: Scaling In Statistical Data Processingmentioning

confidence: 99%

Executable schema mappings for statistical data processing

Atzeni

Bellomarini

Bugiotti

et al. 2017

Distrib Parallel Databases

View full text Add to dashboard Cite

show abstract

“…This approach is way faster than a for loop, where each single iteration only starts when the previous one has been completed. The key point to parallel computing is that the pieces of computations are unrelated (which is the case of some, but not all, loops); examples are evaluating the same R function or fitting the same model on different datasets, which can be particularly useful in simulation studies [34]. The basic approach sets up M different so-called 'worker' processes and splits the main task in M subtasks, which are sent to the workers by a 'master' process.…”

Section: Computationsmentioning

confidence: 99%

Bayesian P-splines and advanced computing in R for a changepoint analysis on spatio-temporal point processes

Altieri

Greco

Illian

et al. 2016

Journal of Statistical Computation and Simulation

View full text Add to dashboard Cite

. (2016) Bayesian P-splines and advanced computing in R for a changepoint analysis on spatio-temporal point processes. Journal of Statistical Computation and Simulation, 86(13), pp. 2531-2545. (doi:10.1080/00949655.2016 This is the author's final accepted version.There may be differences between this version and the published version. You are advised to consult the publisher's version if you wish to cite from it.http://eprints.gla.ac.uk/116178/ This work presents advanced computational aspects of a new method for changepoint detection on spatio-temporal point process data. We summarise the methodology, based on building a Bayesian hierarchical model for the data and declaring prior conjectures on the number and positions of the changepoints, and show how to take decisions regarding the acceptance of potential changepoints. The focus of this work is about choosing an approach that detects the correct changepoint and delivers smooth reliable estimates in a feasible computational time; we propose Bayesian P-splines as a suitable tool for managing spatial variation, both under a computational and a model fitting performance perspective. The main computational challenges are outlined and a solution involving parallel computing in R is proposed and tested on a simulation study. An application is also presented on a dataset of seismic events in Italy over the last 20 years.

show abstract

“…For parallel execution, the foreach package has been adapting the R parallel packages doMC 18 (based on the package multicore on single workstations), doSNOW 19 (based on the package snow with SOCKETS), and doMPI 20 (based on the package Rmpi).…”

Section: Foreach: Foreach Looping Construct For Rmentioning

confidence: 99%

“…18 http://cran.r-project.org/web/packages/doMC/index.html 19 http://cran.r-project.org/web/packages/doSNOW/index.html 20 http://cran.r-project.org/web/packages/doMPI/index.html 21 Luke Tierney is the maintainer of the R package snow.…”

Section: Parallel: Parallel R Packagementioning

confidence: 99%

A Survey of R Software for Parallel Computing

Mahdi

2014

AJAMS

View full text Add to dashboard Cite

This article provides a summary of a selection of some of the high-performance parallel packages (libraries) available from the Comprehensive R Archive Network (CRAN) using the statistical software R. These packages can utilize multicore systems often found in modern personal computers as well as computer cluster or grid computing in order to provide linear speed up the computations in many of advanced statistical modern applications. Some illustrative R parallel codes are given in order to introduce the reader to some basic ideas about parallel programming in R packages. Keywords

show abstract

State of the Art in Parallel Computing withR

Cited by 106 publications

References 26 publications

Executable schema mappings for statistical data processing

Executable schema mappings for statistical data processing

Bayesian P-splines and advanced computing in R for a changepoint analysis on spatio-temporal point processes

A Survey of R Software for Parallel Computing

Contact Info

Product

Resources

About