Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses

Abstract

Due to the upcoming data deluge of genome data, the need for storing and processing large-scale genome data, easy access to biomedical analyses tools, efficient data sharing and retrieval has presented significant challenges. The variability in data volume results in variable computing and storage requirements, therefore biomedical researchers are pursuing more reliable, dynamic and convenient methods for conducting sequencing analyses. This paper proposes a Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses, which enables reliable and highly scalable execution of sequencing analyses workflows in a fully automated manner. Our platform extends the existing Galaxy workflow system by adding data management capabilities for transferring large quantities of data efficiently and reliably (via Globus Transfer), domain-specific analyses tools preconfigured for immediate use by researchers (via user-specific tools integration), automatic deployment on Cloud for on-demand resource allocation and pay-as-you-go pricing (via Globus Provision), a Cloud provisioning tool for auto-scaling (via HTCondor scheduler), and the support for validating the correctness of workflows (via semantic verification tools). Two bioinformatics workflow use cases as well as performance evaluation are presented to validate the feasibility of the proposed approach.

Optimizing computational resource management for the scientific gateways ecosystems based on the service‐oriented paradigm

“…The amount of data generated nowadays, especially in Bioinformatics, brings new problems to be solved in data analysis, management, and storage. If combined with the lack of physical information, it is the road to disaster …”

Architecting cloud‐enabled systems: a systematic survey of challenges and solutions

“…Bux and Leser [PS8] present DynamicCloudSim, which is an extension of CloudSim simulation toolkit to support changes in performance and robustness quality parameters at runtime while scheduling scientific workflows. Liu et al [PS65] present a bioinformatics workflow platform for reliable and highly scalable large scale sequencing analysis. The platform is based on Galaxy workflow system and adds data management capabilities to transfer large quantities of data efficiently and reliably among the processing nodes.…”

Data-aware optimization of bioinformatics workflows in hybrid clouds

“…In the past few years, there is a significant trend in integrating existing tools into unified platforms featuring an abundance of ready to use tools, with particular emphasis on ease of deployment and efficient use of resources of the cloud. A platform based approach is adopted by CloudMan [1], Mercury [38], CLoVR [3], Cloud BioLinux [22] and others [24,32,42,44]. Most of these works are addressing the usability and user friendly aspect of executing bioinformatics workflows, while some of them also support the use of distributed computational resources.…”