Direct data transfer between SOAP web services in orchestration

Subramanian, S.; Sztromwasser, Paweł; Petersen, Kjell; Puntervoll, Pål

doi:10.1145/2428736.2428753

Cited by 6 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data is moved by the services of the workflow directly under the control of the orchestrator in a Data-Flow-Delegation (DFD) (Subramanian et al 2012) and Pipelined-Data-Flow-Delegation (PDFD) approaches (Subramanian et al 2013). In these approaches, the data-flow responsibilities are delegated to the component services of the workflow.…”

Section: In the Perspective Of Data Transfer Optimisationmentioning

confidence: 99%

Reducing data transfer in big-data workflows: the computation-flow delegated approach

Nunes¹,

Deshpande

2019

J. of Data, Inf. and Manag.

View full text Add to dashboard Cite

Existing orchestrated bioinformatics workflow execution approaches necessitate the transfer of datasets from biological data services to the analysis tool (computation) services of the workflow for various data analysis. This model of moving data to computation during workflow execution weakens the performance of the workflow especially when the orchestrated bioinformatics workflow has to handle big-data in it. Since the size of the analysis tools are much smaller than the datasets size in a workflow, in this paper, to minimize the dataflow and improve workflow performance, we propose a novel computation-flow delegated (CFD) approach. The CFD approach lets the tool services of the workflow to dynamically migrate analysis tools towards the datasets to perform computation on data side during workflow execution. We use a set of mobile agents to operate the CFD approach and present a mobile agent-based computation-flow delegation framework (MABCFD) to execute the workflow tasks. We implement the prototype of the MABCFD framework and analyze the performance of the CFD approach empirically by executing in isolation workflow patterns (sequence, fan-out and fan-in) common to bioinformatics applications. Performance analysis shows that the computation-driven CFD approach consistently outperforms the existing data-driven approaches across all patterns and scales favorably with data size.

show abstract

Section: In the Perspective Of Data Transfer Optimisationmentioning

confidence: 99%

Reducing data transfer in big-data workflows: the computation-flow delegated approach

Nunes¹,

Deshpande

2019

J. of Data, Inf. and Manag.

View full text Add to dashboard Cite

show abstract

“…An empirical evaluation of the proposed PDFD vs published DFD given in Subramanian et al (2010Subramanian et al ( , 2012. The DFD approach is briefly described in Section 2.…”

Section: Pipelined Data-flowmentioning

confidence: 99%

“…The organization of the paper follows. Section 2 briefs the regular and the data-flow delegated orchestration given in Subramanian et al (2010Subramanian et al ( , 2012, as a background. Section 3 reports related work.…”

Section: Pipelined Data-flowmentioning

confidence: 99%

“…Although the approach offers improved performance, the data transfer is still indirect and opens privacy concerns when the data storage is owned by a third party. To remedy this, Pipelined data-flow DFD was proposed in Subramanian et al (2010) and applied to orchestration in Subramanian et al (2012), focusing on SOAP Web services. The DFD relieves the orchestrator from mediating data as a middle-man between the component web services, and requires no middle layer between component services and the orchestrator.…”

Section: Related Workmentioning

confidence: 99%

“…To optimize orchestrated workflow performance, in a previous paper (Subramanian et al, 2010) we proposed DFD, which allows data to flow directly from one service to the next. Furthermore, we applied DFD to SOAP Web services in a follow up paper (Subramanian et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Pipelined data‐flow delegated orchestration for data‐intensive eScience workflows

Subramanian

Sztromwasser²,

Puntervoll³

et al. 2013

International Journal of Web Information Systems

Self Cite

View full text Add to dashboard Cite

Purpose -eScience workflows use orchestration for integrating and coordinating distributed and heterogeneous scientific resources, which are increasingly exposed as web services. The rate of growth of scientific data makes eScience workflows data-intensive, challenging existing workflow solutions. Efficient methods of handling large data in scientific workflows based on web services are needed. The purpse of this paper is to address this issue. Design/methodology/approach -In a previous paper the authors proposed Data-Flow Delegation (DFD) as a means to optimize orchestrated workflow performance, focusing on SOAP web services. To improve the performance further, they propose pipelined data-flow delegation (PDFD) for web service-based eScience workflows in this paper, by leveraging from the domain of parallel programming. Briefly, PDFD allows partitioning of large datasets into independent subsets that can be communicated in a pipelined manner. Findings -The results show that the PDFD improves the execution time of the workflow considerably and is capable of handling much larger data than the non-pipelined approach. Practical implications -Execution of a web service-based workflow hampered by the size of data can be facilitated or improved by using services supporting Pipelined Data-Flow Delegation. Originality/value -Contributions of this work include the proposed concept of combining pipelining and Data-Flow Delegation, an XML Schema supporting the PDFD communication between services, and the practical evaluation of the PDFD approach.

show abstract

Big-Data Approaches for Bioinformatics Workflows: A Comparative Assessment

Nunes

Deshpande

2016

Communications in Computer and Information Science

View full text Add to dashboard Cite

Direct data transfer between SOAP web services in orchestration

Cited by 6 publications

References 25 publications

Reducing data transfer in big-data workflows: the computation-flow delegated approach

Reducing data transfer in big-data workflows: the computation-flow delegated approach

Pipelined data‐flow delegated orchestration for data‐intensive eScience workflows

Big-Data Approaches for Bioinformatics Workflows: A Comparative Assessment

Contact Info

Product

Resources

About