A novel graph based approach for automatic composition of high quality grid workflows

Abstract

The workflow paradigm is one of the most important programming models for the Grid. The composition of Grid workflows has been widely studied in the Grid community. However, there is still a lack of a general and efficient approach for automatic composition of Grid workflows. In this paper, we present a STRIPS (Stanford Research Institute Problem Solver) based formal definition of the Grid workflow composition problem, followed by a novel graph based algorithm for automatic composition of high quality (portable, fault tolerant and optimized) Grid workflows. Our algorithm searches for semantic descriptions of workflow activities, i.e., Activity Functions (AFs), defined by ontologies and composes them into Grid workflows using AF Data Dependence (ADD) graphs. The composition process consists of three phases: ADD graph creation, workflow extraction, and workflow optimization. The worst case complexity of our algorithm is quadratic in the number of AFs. An extension of our algorithm to compose Grid workflows with branches and loops is also presented. Experimental results illustrate the effectiveness and efficiency of our approach: (i) the measured worst case execution time of our algorithm further proofs the analyzed time complexity; (ii) the composition of the real world meteorology Grid workflow application MeteoAG with our algorithm takes approximate half a second; and (iii) the execution time of the MeteoAG workflow when running on the Austrian Grid is reduced by up to 25% and the speedup is increased by up to 2.24 by applying our workflow optimization techniques.

Scientific workflows: Past, present and future

“…The ENTICE project, http://www.entice-project.eu/, enriched ASKALON with multi-objective optimised placement of fragmented VMs from a distributed repository with faster delivery, enhanced availability, and reduced storage costs [42]. ASKALON provides automatic composition of workflows using semantic descriptions of workflow activities and ontologies; delivering resilience by creating alternative workflows or sub-graphs [44]. continued on next page continued from previous page Advances and successes in the past 10 years WFS Advances or major achievements dispel4py The dispel4py system was developed to enable seismologists to exploit their growing wealth of data -two examples are: Seismic Ambient Noise Cross-Correlation and MISFIT calculation.…”