A multi-dimensional classification model for scientific workflow characteristics

Ramakrishnan, Lavanya; Plale, Beth

doi:10.1145/1833398.1833402

Cited by 38 publications

(32 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Work in this area is primarily focused on outlining the inventory of workflow development constructs provided by different workflow languages and the ways of combining those constructs. Classification models have also been developed to detect additional patterns in structure, usage and data [13]. Scientific workflows are characterized by the lack of complex control constructs, where the order of execution is determined by the availability of data.…”

Section: Related Workmentioning

confidence: 99%

Common motifs in scientific workflows: An empirical analysis

Garijo

Alper

Belhajjame

et al. 2012

2012 IEEE 8th International Conference on E-Science

View full text Add to dashboard Cite

Abstract-While workflow technology has gained momentum in the last decade as a means for specifying and enacting computational experiments in modern science, reusing and repurposing existing workflows to build new scientific experiments is still a daunting task. This is partly due to the difficulty that scientists experience when attempting to understand existing workflows, which contain several data preparation and adaptation steps in addition to the scientifically significant analysis steps. One way to tackle the understandability problem is through providing abstractions that give a high-level view of activities undertaken within workflows. As a first step towards abstractions, we report in this paper on the results of a manual analysis performed over a set of real-world scientific workflows from Taverna and Wings systems. Our analysis has resulted in a set of scientific workflow motifs that outline i) the kinds of data intensive activities that are observed in workflows (data oriented motifs), and ii) the different manners in which activities are implemented within workflows (workflow oriented motifs). These motifs can be useful to inform workflow designers on the good and bad practices for workflow development, to inform the design of automated tools for the generation of workflow abstractions, etc.

show abstract

Section: Related Workmentioning

confidence: 99%

Common motifs in scientific workflows: An empirical analysis

Garijo

Alper

Belhajjame

et al. 2012

2012 IEEE 8th International Conference on E-Science

View full text Add to dashboard Cite

show abstract

“…Using the Pegasus APIs, a wordcount workflow application composed of four jobs is written. This workflow has both control and data dependencies [42] among its jobs along with the split and merge characteristics, which are common characteristics in scientific workflows. The first job (Split job) takes a text file and splits it into two files of almost equal length.…”

Section: Resultsmentioning

confidence: 99%

Re-provisioning of Cloud-Based Execution Infrastructure Using the Cloud-Aware Provenance to Facilitate Scientific Workflow Execution Reproducibility

Hasham

Munir

McClatchey

et al. 2016

Communications in Computer and Information Science

View full text Add to dashboard Cite

Abstract. Provenance has been considered as a means to achieve scientific workflow reproducibility to verify the workflow processes and results. Cloud computing provides a new computing paradigm for the workflow execution by o↵ering a dynamic and scalable environment with on-demand resource provisioning. In the absence of Cloud infrastructure information, achieving workflow reproducibility on the Cloud becomes a challenge. This paper presents a framework, named ReCAP, to capture the Cloud infrastructure information and to interlink it with the workflow provenance to establish the Cloud-Aware Provenance (CAP). This paper identifies di↵erent scenarios of using the Cloud for workflow execution and presents di↵erent mapping approaches. The reproducibility of the workflow execution is performed by re-provisioning the similar Cloud resources using CAP and re-executing the workflow; and by comparing the outputs of workflows. Finally, this paper also presents the evaluation of ReCAP in terms of captured provenance, workflow execution time and workflow output comparison.

show abstract

“…Classification models have been developed to detect additional patterns in structure, usage and data [22]. More high-level patterns, associated to specific cases of use (data curation, analysis) have been identified in Taverna and Wings workflows [23].…”

Section: Related Workmentioning

confidence: 99%

Distilling structure in Taverna scientific workflows: a refactoring approach

et al. 2014

View full text Add to dashboard Cite

BackgroundScientific workflows management systems are increasingly used to specify and manage bioinformatics experiments. Their programming model appeals to bioinformaticians, who can use them to easily specify complex data processing pipelines. Such a model is underpinned by a graph structure, where nodes represent bioinformatics tasks and links represent the dataflow. The complexity of such graph structures is increasing over time, with possible impacts on scientific workflows reuse. In this work, we propose effective methods for workflow design, with a focus on the Taverna model. We argue that one of the contributing factors for the difficulties in reuse is the presence of "anti-patterns", a term broadly used in program design, to indicate the use of idiomatic forms that lead to over-complicated design. The main contribution of this work is a method for automatically detecting such anti-patterns, and replacing them with different patterns which result in a reduction in the workflow's overall structural complexity. Rewriting workflows in this way will be beneficial both in terms of user experience (easier design and maintenance), and in terms of operational efficiency (easier to manage, and sometimes to exploit the latent parallelism amongst the tasks).ResultsWe have conducted a thorough study of the workflows structures available in Taverna, with the aim of finding out workflow fragments whose structure could be made simpler without altering the workflow semantics. We provide four contributions. Firstly, we identify a set of anti-patterns that contribute to the structural workflow complexity. Secondly, we design a series of refactoring transformations to replace each anti-pattern by a new semantically-equivalent pattern with less redundancy and simplified structure. Thirdly, we introduce a distilling algorithm that takes in a workflow and produces a distilled semantically-equivalent workflow. Lastly, we provide an implementation of our refactoring approach that we evaluate on both the public Taverna workflows and on a private collection of workflows from the BioVel project.ConclusionWe have designed and implemented an approach to improving workflow structure by way of rewriting preserving workflow semantics. Future work includes considering our refactoring approach during the phase of workflow design and proposing guidelines for designing distilled workflows.

show abstract

A multi-dimensional classification model for scientific workflow characteristics

Cited by 38 publications

References 11 publications

Common motifs in scientific workflows: An empirical analysis

Common motifs in scientific workflows: An empirical analysis

Re-provisioning of Cloud-Based Execution Infrastructure Using the Cloud-Aware Provenance to Facilitate Scientific Workflow Execution Reproducibility

Distilling structure in Taverna scientific workflows: a refactoring approach

Contact Info

Product

Resources

About