Workflow-Level Parametric Study Support by MOTEUR and the P-GRADE Portal

Glatard, Tristan; Sipos, Gergely; Montagnat, Johan; Farkas, Zoltán; Kacsuk, Péter

doi:10.1007/978-1-84628-757-2_18

Cited by 11 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, the concept of Web portals for science is commonplace in distributed computing. Several SWfMS [15,40,41] solutions offer science portals that allow users to submit and monitor their workflow execution in distributed environments such as grids and clouds.…”

Section: Interface For Interactionmentioning

confidence: 99%

Dynamic steering of HPC scientific workflows: A survey

Mattoso

Dias

Ocaña

et al. 2015

Future Generation Computer Systems

View full text Add to dashboard Cite

Section: Interface For Interactionmentioning

confidence: 99%

Dynamic steering of HPC scientific workflows: A survey

Mattoso

Dias

Ocaña

et al. 2015

Future Generation Computer Systems

View full text Add to dashboard Cite

“…However, they are not natively interfaced to a Grid infrastructure and they require adaptations to exploit parallel Grid resources. The P-GRADE portal [18] workflow manager is a Grid-enabled workflow manager which aims at exploiting both approaches: originally interfaced to DAGMan, it has recently been extended to support so called parametric studies [15]. A parametric workflow is a template-based workflow for which multiple tasks will be instantiated similarly to the way the gLite parametric jobs produce multiple task executions.…”

Section: Scientific Workflows Descriptionmentioning

confidence: 99%

Workflow-Based Data Parallel Applications on the EGEE Production Grid Infrastructure

et al. 2008

Self Cite

View full text Add to dashboard Cite

Setting up and deploying complex applications on a Grid infrastructure is still challenging and the programming models are rapidly evolving. Efficiently exploiting Grid parallelism is often not straight forward. In this paper, we report on the techniques used for deploying applications on the EGEE production Grid through four experiments coming from completely different scientific areas: J. Montagnat I3S laboratory, CNRS, Sophia Antipolis, France J. Montagnat (B) EPU, RAINBOW, 930 route des Colles, nuclear fusion, astrophysics and medical imaging. These applications have in common the need for manipulating huge amounts of data and all are computationally intensive. All the cases studied show that the deployment of data intensive applications require the development of more or less elaborated application-level workload management systems on top of the gLite middleware

show abstract

“…Glatard et al [19] define two workflow manager architectures: task-based, where workflow manager is responsible in handling computing task; service-based, where computation is handled by external services. MOTEUR [18], is a workflow enactor that optimises service-based workflows by exploiting intrinsic parallelism (no dependencies between services), data parallelism (different data on different threads) and service parallelism (pipe-lining different services in handling independent data sets).…”

Section: Related Workmentioning

confidence: 99%

Towards optimising distributed data streaming graphs using parallel streams

Liew

Atkinson

Hemert

et al. 2010

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

View full text Add to dashboard Cite

Modern scientific collaborations have opened up the opportunity of solving complex problems that involve multidisciplinary expertise and large-scale computational experiments. These experiments usually involve large amounts of data that are located in distributed data repositories running various software systems, and managed by different organisations. A common strategy to make the experiments more manageable is executing the processing steps as a workflow. In this paper, we look into the implementation of fine-grained data-flow between computational elements in a scientific workflow as streams. We model the distributed computation as a directed acyclic graph where the nodes represent the processing elements that incrementally implement specific subtasks. The processing elements are connected in a pipelined streaming manner, which allows task executions to overlap. We further optimise the execution by splitting pipelines across processes and by introducing extra parallel streams. We identify performance metrics and design a measurement tool to evaluate each enactment. We conducted experiments to evaluate our optimisation strategies with a real world problem in the Life Sciences-EURExpress-II. The paper presents our distributed data-handling model, the optimisation and instrumentation strategies and the evaluation experiments. We demonstrate linear speed up and argue that this use of data-streaming to enable both overlapped pipeline and parallelised enactment is a generally applicable optimisation strategy.

show abstract

Workflow-Level Parametric Study Support by MOTEUR and the P-GRADE Portal

Cited by 11 publications

References 0 publications

Dynamic steering of HPC scientific workflows: A survey

Dynamic steering of HPC scientific workflows: A survey

Workflow-Based Data Parallel Applications on the EGEE Production Grid Infrastructure

Towards optimising distributed data streaming graphs using parallel streams

Contact Info

Product

Resources

About