Building gateways for life-science applications using the dynamic application runtime environment (DARE) framework

Kim, Joohyun; Maddineni, Sharath; Jha, Shantenu

doi:10.1145/2016741.2016782

Cited by 5 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DARE and beyond: PMR-based NGS tools, implemented and scrutinized in this work, were developed in conjunction with our development of the runtime environment for dynamic applications, Distributed Application Runtime Environment (DARE) [31,32]. DARE is a strategically important component for the development of Science Gateway for NGS data analytics and downstream analysis.…”

Section: Discussionmentioning

confidence: 99%

Understanding mapreduce-based next-generation sequencing alignment on distributed cyberinfrastructure

Mantha

Kim

Luckow

et al. 2012

Proceedings of the 3rd International Workshop on Emerging Computational Methods for the Life Sciences

Self Cite

View full text Add to dashboard Cite

Although localization of Next-Generation Sequencing (NGS) data is suitable for many analysis and usage scenarios, it is not universally desirable, nor possible. However most solutions "impose" the localization of data as a pre-condition for NGS analytics. We analyze several existing tools and techniques that use MapReduce programming model for NGS data analysis to determine their effectiveness and extensibility to support distributed data scenarios. We find limitations at multiple levels. To overcome these limitations, we developed a Pilot-based MapReduce (PMR) -which is a novel implementation of MapReduce using a Pilot task and data management implementation. PMR provides an effective means by which a variety of new or existing methods for NGS and downstream analysis can be carried out whilst providing efficiency and scalability across multiple clusters. Pilot-MapReduce (PMR) circumvents the use of global reduce and yet provides an effective, scalable and distributed solution for MapReduce programming model. We compare and contrast the PMR approach to similar capabilities of Seqal and Crossbow, two other tools which are based on conventional Hadoop-based MapReduce for NGS reads alignment and duplicate read removal or SNP finding, respectively. We find that PMR is a viable tool to support distributed NGS analytics, particularly providing a framework that supports parallelism at multiple levels.

show abstract

Section: Discussionmentioning

confidence: 99%

Understanding mapreduce-based next-generation sequencing alignment on distributed cyberinfrastructure

Mantha

Kim

Luckow

et al. 2012

Proceedings of the 3rd International Workshop on Emerging Computational Methods for the Life Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…To achieve the goal of supporting both types of parallelism on heterogeneous distributed computing resources, we have developed the DARE framework .…”

Section: Ngs Data Analytics Using Distributed Scalable Architecturesmentioning

confidence: 99%

“…The DARE framework , is a runtime infrastructure that has been shown to enable the seamless utilization of heterogeneous distributed computing resources , including high‐performance computing (HPC) grids and cloud environments.…”

Section: Introductionmentioning

confidence: 99%

Advancing next‐generation sequencing data analytics with scalable distributed infrastructure

Kim

Maddineni

Jha

2013

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SUMMARYWith the emergence of popular next-generation sequencing (NGS)-based genome-wide protocols such as chromatin immunoprecipitation followed by sequencing (ChIP-Seq) and RNA-Seq, there is a growing need for research and infrastructure to support the requirement of effectively analyzing NGS data. Such research and infrastructure do not replace but complement algorithmic advances developments in analyzing NGS data. We present a runtime environment, Distributed Application Runtime Environment, that supports the scalable, flexible, and extensible composition of capabilities that cover the primary requirements of NGSbased analytics. In this work, we use BFAST as a representative stand-alone tool used for NGS data analysis and a ChIP-Seq pipeline as a representative pipeline-based approach to analyze the computational requirements. We analyze the performance characteristics of BFAST and understand its dependency on different input parameters. The computational complexity of genome-wide mapping using BFAST, amongst other factors, depends upon the size of a reference genome and the data size of short reads. Characterizing the performance suggests that the mapping benefits from both scaling-up (increased fine-grained parallelism) and scaling-out (task-level parallelism -local and distributed). For certain problem instances, scaling-out can be a more efficient approach than scaling-up. On the basis of investigations using the pipeline for ChIP-Seq, we also discuss the importance of dynamical execution of tasks.

show abstract

“…Software projects such as those reported in [12], [13] and [14] support applications on the cloud, all of which require the user to have extensive knowledge of the cloud dashboard to be able to port an existing analytical workload onto the cloud. The options provided by such projects for a fully configurable cloud cluster can fit well with the skill set of a cloud developer, thereby narrowing their wide usage.…”

Section: Introductionmentioning

confidence: 99%

RBioCloud: A Light-Weight Framework for Bioconductor and R-based Jobs on the Cloud

Varghese

Patel

Barker

2015

IEEE/ACM Trans. Comput. Biol. and Bioinf.

View full text Add to dashboard Cite

Abstract-Large-scale ad hoc analytics of genomic data is popular using the R-programming language supported by over 700 software packages provided by Bioconductor. More recently, analytical jobs are benefitting from on-demand computing and storage, their scalability and their low maintenance cost, all of which are offered by the cloud. While Biologists and Bioinformaticists can take an analytical job and execute it on their personal workstations, it remains challenging to seamlessly execute the job on the cloud infrastructure without extensive knowledge of the cloud dashboard. How analytical jobs can not only with minimum effort be executed on the cloud, but also how both the resources and data required by the job can be managed is explored in this paper. An open-source light-weight framework for executing R-scripts using Bioconductor packages, referred to as 'RBioCloud', is designed and developed. RBioCloud offers a set of simple command-line tools for managing the cloud resources, the data and the execution of the job. Three biological test cases validate the feasibility of RBioCloud. The framework is available from http://www.rbiocloud.com.

show abstract

Building gateways for life-science applications using the dynamic application runtime environment (DARE) framework

Cited by 5 publications

References 19 publications

Understanding mapreduce-based next-generation sequencing alignment on distributed cyberinfrastructure

Understanding mapreduce-based next-generation sequencing alignment on distributed cyberinfrastructure

Advancing next‐generation sequencing data analytics with scalable distributed infrastructure

RBioCloud: A Light-Weight Framework for Bioconductor and R-based Jobs on the Cloud

Contact Info

Product

Resources

About