noWorkflow: Capturing and Analyzing Provenance of Scripts

Murta, Leonardo; Braganholo, Vanessa; Chirigati, Fernando; Koop, David; Freire, Juliana

doi:10.1007/978-3-319-16462-5_6

Cited by 87 publications

(103 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The suggestions are mined from retrospective provenance data 1 [1,31]. (Leake and Kendall-Morwick [13] refer to retrospective provenance as "execution traces, " terminology also adopted in other works [32]). The reason for this is that execution traces are simpler than the workflow structure, since they are basically sequential.…”

Section: Approaches Based On Case-based Reasoningmentioning

confidence: 99%

Improving workflow design by mining reusable tasks

Tosta¹,

Braganholo

Murta

et al. 2015

J Braz Comput Soc

Self Cite

View full text Add to dashboard Cite

Background: With the increasing popularity of scientific workflow management systems (SWfMS), more and more workflow specifications are becoming available. Such specifications contain precious knowledge that can be reused to produce new workflows. It is a fact that provenance data can help reusing third party code. However, finding the dependencies among programs without the support of a tool is not a trivial activity and, in many cases, becomes a barrier to build more sophisticated models and analysis. Due to the huge number of task versions available and their configuration parameters, this activity is highly error prone and counterproductive. Methods: In this work, we propose workflow recommender (WR), a recommendation service that aims at suggesting frequent combinations of workflow tasks for reuse. It works similarly to an e-commerce application that applies data mining techniques to help users find items they would like to purchase, predicting a list based on other user's choices. Results: Our experiments show that our approach is effective both in terms of performance and precision of the results. Conclusions:The approach is general in the sense that it can be coupled to any SWfMS.

show abstract

Section: Approaches Based On Case-based Reasoningmentioning

confidence: 99%

Improving workflow design by mining reusable tasks

Tosta¹,

Braganholo

Murta

et al. 2015

J Braz Comput Soc

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the architecture discussion, three papers were presented: noWorkflow [80] is a tool that transparently captures provenance of scripts and enables reproducibility, LabelFlow [46] uses domain-specific labels with workflow provenance as a platform for data artifacts labelling; and [84] proposes software provenance to be included as part of software packages.…”

Section: Provenancementioning

confidence: 99%

Provenance in Open Data Entity-Centric Aggregation

Giunchiglia

Reyad

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

An increasing number of web services these days require combining data from several data providers into an aggregated database. Usually this aggregation is based on the linked data approach. On the other hand, the entity-centric model is a promising data model that outperforms the linked data approach because it solves the lack of explicit semantics and the semantic heterogeneity problems. However, current open data which is available on the web as raw datasets can not be used in the entity-centric model before processing them with an import process to extract the data elements and insert them correctly in the aggregated entity-centric database. It is essential to certify the quality of these imported data elements, especially the background knowledge part which acts as input to semantic computations, because the quality of this part affects directly the quality of the web services which are built on top of it. Furthermore, the aggregation of entities and their attribute values from different sources raises three problems: the need to trace the source of each element, the need to trace the links between entities which can be considered equivalent and the need to handle possible conflicts between different values when they are imported from various data sources. In this thesis, we introduce a new model to certify the quality of a back ground knowledge base which separates linguistic and language independent elements. We also present a pipeline to import entities from open data repositories to add the missing implicit semantics and to eliminate the semantic heterogeneity. Finally, we show how to trace the source of attribute values coming from different data providers; how to choose a strategy for handling possible conflicts between these values; and how to keep the links between identical entities which represent the same real world entity.

show abstract

“…One motivation is obtaining a provenance abstraction capturing data's flow or the process's dependencies. In noWorkflow [24] authors analyse Python scripts to extract function-call hierarchies, which they use to create a view over provenance traces collected by run-time instrumentation of the functions' reads and writes to the file system. In [25] authors employ a taint tracking framework, which instruments programme executions and records which computations are affected by tainted data sources.…”

Section: Related Workmentioning

confidence: 99%

Static analysis of Taverna workflows to predict provenance patterns

Alper

Belhajjame

Goble

2017

Future Generation Computer Systems

View full text Add to dashboard Cite

Workflows have found adoption in scientific domains particularly due to their automation and provenance features. Using workflows scientists can repeat analyses with different input parameters to later use provenance to access and compare results based on respective parameters. An assumption that is often made is that by designing an analysis as a workflow, we get parameterto-result traceability for free with workflow provenance. This assumption holds for cases of coarse-grained traceability, where an entire workflow is subjected to repetition and all workflow parameters contribute to all results. On the other hand for cases requiring finer grained traceability, where a workflow is configured with collections of parameters and analyses within a workflow are repeated with combinations of parameters from collections, this assumption is not guaranteed to hold. In this paper we identify two dimensions that affect fine-grained traceability as: 1) the level of granularity supported by a workflow system in modelling parameters/data in workflows and in provenance, which we name as the level of support for Factorial Design, and 2) the practice of scientists in successfully encoding Factorial Design into workflows. Taverna is a workflow system that provides extensive features for factorial design, meanwhile it provides an uncontrolled approach to workflow design; meaning scientists may create workflows, which, when run, could break traceability in provenance. Using a real-world Taverna workflow we show how broken traceability manifests in provenance and how it can render provenance practically useless for accessing workflow outputs derived from particular input parameters. In order to prevent broken traceability from occurring, we describe a rule-based static analysis technique, which operates over workflow descriptions and anticipates patterns in provenance. Our rules exploit the well-defined execution behaviour in the Taverna system. In order to understand Factorial Design support in workflow systems in general, we provide a comparative survey. We conclude that other workflow systems also provide constructs for Factorial Design, and, similar to Taverna, they too are prone to broken traceability.

show abstract

noWorkflow: Capturing and Analyzing Provenance of Scripts

Cited by 87 publications

References 14 publications

Improving workflow design by mining reusable tasks

Improving workflow design by mining reusable tasks

Provenance in Open Data Entity-Centric Aggregation

Static analysis of Taverna workflows to predict provenance patterns

Contact Info

Product

Resources

About