FragFlow Automated Fragment Detection in Scientific Workflows

Abstract

Abstract-Scientific workflows provide the means to define, execute and reproduce computational experiments. However, reusing existing workflows still poses challenges for workflow designers. Workflows are often too large and too specific to reuse in their entirety, so reuse is more likely to happen for fragments of workflows. These fragments may be identified manually by users as sub-workflows, or detected automatically. In this paper we present the FragFlow approach, which detects workflow fragments automatically by analyzing existing workflow corpora with graph mining algorithms. FragFlow detects the most common workflow fragments, links them to the original workflows and visualizes them. We evaluate our approach by comparing FragFlow results against user-defined sub-workflows from three different corpora of the LONI Pipeline system. Based on this evaluation, we discuss how automated workflow fragment detection could facilitate workflow reuse.

Will AI Write Scientific Papers in the Future?

“…Semantic workflows also enable us to use machine learning to detect common workflow fragments that scientists use with different data, and we demonstrated this with a large collection of neuroimaging workflows (Garijo et al 2014). This is work with Daniel Garijo here at USC and Oscar Corcho of the Polytechnic University of Madrid.…”

Will AI write scientific papers in the future?

“…Semantic workflows also enable us to use machine learning to detect common workflow fragments that scientists use with different data, and we demonstrated this with a large collection of neuroimaging workflows (Garijo et al 2014). This is work with Daniel Garijo here at USC and Oscar Corcho of the Polytechnic University of Madrid.…”

A novel transversal processing model to build environmental big data services in the cloud

“…In turn, the solutions that reuse workflow fragments for the composition of new solutions are mainly focused on algorithms that search for fragments of tasks within workflow repositories, that are suitable to be reused when building new solutions [61] [13]. Some of these works produce the automatic design of solutions by using fragments of workflows, but there are still limitations and areas of opportunity of this type of approaches that the model proposed in this paper supports.…”