Data partitioning enables the use of standard SOAP Web Services in genome-scale workflows

Abstract

SummaryBiological databases and computational biology tools are provided by research groups around the world, and made accessible on the Web. Combining these resources is a common practice in bioinformatics, but integration of heterogeneous and often distributed tools and datasets can be challenging. To date, this challenge has been commonly addressed in a pragmatic way, by tedious and error-prone scripting. Recently however a more reliable technique has been identified and proposed as the platform that would tie together bioinformatics resources, namely Web Services. In the last decade the Web Services have spread wide in bioinformatics, and earned the title of recommended technology. However, in the era of high-throughput experimentation, a major concern regarding Web Services is their ability to handle large-scale data traffic. We propose a stream-like communication pattern for standard SOAP Web Services, that enables efficient flow of large data traffic between a workflow orchestrator and Web Services. We evaluated the data-partitioning strategy by comparing it with typical communication patterns on an example pipeline for genomic sequence annotation. The results show that data-partitioning lowers resource demands of services and increases their throughput, which in consequence allows to execute in-silico experiments on genome-scale, using standard SOAP Web Services and workflows. As a proof-of-principle we annotated an RNA-seq dataset using a plain BPEL workflow engine.

Heart Cycle: Facilitating the deployment of advanced care processes

“…There are attempts to standardize care using knowledge-based systems such as GLIF [3], [5] Asbru [6] or ProFORMA [7], which represent medical knowledge in a declarative way, defining rules that describe the process. Nevertheless, its execution is complex [8] and requires additional advanced technical skills that doctors are not familiar with, to be effectively used.…”

“…To enable this, the implementation pattern recognition algorithms that allows a human understandable presentation of the process deployment is required. In fact, it is demonstrated that the use experts in the middle of the pattern recognition process improves significantly the accuracy of the models [8]. Therefore, the use of pattern recognition models that represents the whole process in an expert understandable way constitutes an accurate way to support care optimization processes.…”

Pipelined data‐flow delegated orchestration for data‐intensive eScience workflows

“…Taverna (Oinn et al, 2004), an open source and domain-independent workflow management system uses superscalar pipelining when the input to a workflow is a list of data elements, and streaming pipelines if the input is a stream of discrete elements (Sroka et al, 2010). Similarly, in Sztromwasser et al (2011), authors proposed a stream-like communication pattern for SOAP Web services in the process of regular orchestration. In Triana (Majithia et al, 2004), an open source problem solving environment, the CPU-intensive components of the workflow can be grouped by a user for parallel processing on data-segments (Taylor et al, 2007).…”

“…The DFD relieves the orchestrator from mediating data as a middle-man between the component web services, and requires no middle layer between component services and the orchestrator. In this paper, we present PDFD orchestration by combining the advantages of the stream-like communication pattern presented in Sztromwasser et al (2011) and the DFD, for orchestrating SOAP Web services.…”

Reducing data transfer in big-data workflows: the computation-flow delegated approach

“…Biological databases and analysis tools (a.k.a computational/bioinformatics tools) are important resources in bioinformatics experiments. Combining these resources and performing computation is a common practice in bioinformatics (Sztromwasser et al 2011). In recent years, bioinformatics experiments are generating large volumes of data and this is increasing the sizes of biological databases which seriously impedes bioinformatics experiments.…”