A lightweight method for evaluating in situ workflow efficiency

Anh, Tu Mai; Pottier, Loïc; Caíno‐Lores, Silvina; Silva, Rafael Ferreira da; Cuendet, Michel A.; Weinstein, Harel; Estrada, Trilce; Taufer, Michela; Deelman, Ewa

doi:10.1016/j.jocs.2020.101259

Cited by 9 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This means that the time to execute each stage remains constant over all the ρ steps of the application. This hypothesis holds when the number of steps is large enough (ρ ≥ 3), and the impact of warming-up steps is negligible [10].…”

Section: Modeling the Examinimd In-situ Workflowmentioning

confidence: 94%

“…Different approaches have been proposed in the literature to ascertain the performance gains brought by an in-situ (or in-transit) execution of a given scientific workflow application and determine the best configuration deployment of its components on a given target platform. We distinguish these approaches depending on whether they rely on actual experiments [3][4][5][6][7] or resort to simulation [8][9][10] to evaluate the performance of in-situ workflows. The former is intrinsically time-and resource-consuming while the latter may suffer from simplification biases when the abstract versions of the in-situ workflow components are developed.…”

Section: Related Workmentioning

confidence: 99%

“…The simulator used a predetermined set of simulation parameters to study the impact of the in-situ analyses that are scheduled on the performance of the entire in-situ execution. In a previous work [10], we extrapolated benchmarking performance of realistic MD simulation engines to create a synthetic MD simulation that emulates the behavior of MD simulations while using fewer computing resources. Our proposed synthetic MD simulation considered computation units as timing delays without actually performing any computing operations.…”

Section: Related Workmentioning

confidence: 99%

“…The following execution model is adapted from the one proposed in [10]. The main difference lies in the data transfer pattern between the simulation and analytics components.…”

Section: Modeling the Examinimd In-situ Workflowmentioning

confidence: 99%

“…The main difference lies in the data transfer pattern between the simulation and analytics components. In the model in [10], the simulation component only sends data to the analytics once, with some buffer constraints imposed by the DTL, while in the considered in-situ workflow, the the results of analytics component have to be received before starting a new analysis. Such a model can help users to obtain a first approximation of the impact of the different configuration parameters of the in-situ workflow (i.e., cost and frequency of analysis) and of the allocation and mapping scheme (i.e., number of nodes and core allocation ratio) on the performance of the application.…”

Section: Modeling the Examinimd In-situ Workflowmentioning

confidence: 99%

See 4 more Smart Citations

SIM-SITU: A Framework for the Faithful Simulation of in-situ Workflows

Honoré¹,

Do²,

Pottier³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The amount of data generated by numerical simulations in various scientific domains such as molecular dynamics, climate modeling, biology, or astrophysics, led to a fundamental redesign of application workflows. The throughput and the capacity of storage subsystems have not evolved as fast as the computing power in extreme-scale supercomputers. As a result, the classical posthoc analysis of simulation outputs became highly inefficient. In-situ workflows have then emerged as a solution in which simulation and data analytics are intertwined through shared computing resources, thus lower latencies.Determining the best allocation, i.e., how many resources to allocate to each component of an in-situ workflow; and mapping, i.e., where and at which frequency to run the data analytics component, is a complex task whose performance assessment is crucial to the efficient execution of in-situ workflows. However, such a performance evaluation of different allocation and mapping strategies usually relies either on directly running them on the targeted execution environments, which can rapidly become extremely time-and resource-consuming, or on resorting to the simulation of simplified models of the components of an in-situ workflow, which can lack of realism. In both cases, the validity of the performance evaluation is limited.To address this issue, we introduce Sim-Situ, a framework for the faithful simulation of insitu workflows. This framework builds on the SimGrid toolkit and benefits of several important features of this versatile simulation tool. We designed Sim-Situ to reflect the typical structure of in-situ workflows and thanks to its modular design, Sim-Situ has the necessary flexibility to easily and faithfully evaluate the behavior and performance of various allocation and mapping strategies for in-situ workflows. We illustrate the simulation capabilities of Sim-Situ on a Molecular Dynamics use case. We study the impact of different allocation and mapping strategies on performance and show how users can leverage Sim-Situ to determine interesting tradeoffs when designing their in-situ workflow.

show abstract

Section: Modeling the Examinimd In-situ Workflowmentioning

confidence: 94%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…The following execution model is adapted from the one proposed in [10]. The main difference lies in the data transfer pattern between the simulation and analytics components.…”

Section: Modeling the Examinimd In-situ Workflowmentioning

confidence: 99%

Section: Modeling the Examinimd In-situ Workflowmentioning

confidence: 99%

See 3 more Smart Citations

SIM-SITU: A Framework for the Faithful Simulation of in-situ Workflows

Honoré¹,

Do²,

Pottier³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Performance assessment of ensembles of in situ workflows under resource constraints

Anh

Pottier

Silva

et al. 2022

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Summary Scientific breakthroughs in biomolecular methods and improvements in hardware technology have shifted from a long‐running simulation to a large set of shorter simulations running simultaneously, called an ensemble. In an ensemble, simulations are usually coupled with analyses of data produced by the simulations. In situ methods can be used to analyze large volumes of data generated by scientific simulations at runtime (i.e., simulations and analyses are performed concurrently). In this work, we study the execution of ensemble‐based simulations paired with in situ analyses using in‐memory staging methods. Using an ensemble of molecular dynamics in situ workflows with multiple simulations and analyses, we first show that collecting traditional metrics such as makespan, instructions per cycle, memory usage, or cache miss ratio is not sufficient to characterize complex behaviors of ensembles. We propose a method to evaluate the performance of ensembles of workflows that captures multiple resource usage aspects: resource efficiency, resource allocation, and resource provisioning. Experimental results demonstrate that the proposed method can effectively distinguish the performance of different component placements in an ensemble with up to 32 ensemble members. By evaluating different co‐location scenarios, our proposed performance indicators demonstrate benefits of co‐locating simulation and coupled analyses within a compute node.

show abstract

Efficient in-situ workflow planning for geographically distributed heterogeneous environments

Song

2023

Future Generation Computer Systems

View full text Add to dashboard Cite

A lightweight method for evaluating in situ workflow efficiency

Cited by 9 publications

References 19 publications

SIM-SITU: A Framework for the Faithful Simulation of in-situ Workflows

SIM-SITU: A Framework for the Faithful Simulation of in-situ Workflows

Performance assessment of ensembles of in situ workflows under resource constraints

Efficient in-situ workflow planning for geographically distributed heterogeneous environments

Contact Info

Product

Resources

About