Evaluating Distributed Execution of Workloads

Turilli, Matteo; Babuji, Yadu; Merzky, André; Ha, Ming Tai; Wilde, Michael; Jha, Shantenu

doi:10.1109/escience.2017.41

Cited by 11 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1, purple α-δ). The distributed scheduling capabilities of RADICAL-Pilot offered the possibility to minimize the time to completion of tasks execution, obtaining both qualitative and quantitative improvements [16]. Qualitatively, RADICAL-Cybertools enabled Swift to execute workflows concurrently on both HPC and HTC resources via late binding of both tasks to pilots and pilots to resources.…”

Section: A Integrating End-to-end Workflow Systemsmentioning

confidence: 99%

Middleware Building Blocks for Workflow Systems

et al. 2019

Self Cite

View full text Add to dashboard Cite

This paper describes a building blocks approach to the design of scientific workflow systems. We discuss RADICAL-Cybertools as one implementation of the building blocks concept, showing how they are designed and developed in accordance with this approach. This paper offers three main contributions: (i) showing the relevance of the design principles underlying the building blocks approach to support scientific workflows on high performance computing platforms; (ii) illustrating a set of building blocks that enable multiple points of integration, "unifying" conceptual reasoning across otherwise very different tools and systems; and (iii) case studies discussing how RADICAL-Cybertools are integrated with existing workflow, workload, and general purpose computing systems and used to develop domainspecific workflow systems.

show abstract

Section: A Integrating End-to-end Workflow Systemsmentioning

confidence: 99%

Middleware Building Blocks for Workflow Systems

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…RCT are a testbed for engineering research, mostly focused on foundational abstractions [21], architectural paradigms [6], application patterns [15,4], and performance analysis of distributed middleware on diverse computing infrastructures [21,19]. Among the most representative projects supported by RP as a standalone system, the Abstractions and Integrated Middleware for Extreme-Scale Science (AIMES) project enabled extremescale distributed computing via dynamic federation of heterogeneous computing infrastructures.…”

Section: Illustrative Examplesmentioning

confidence: 99%

RADICAL-Cybertools: Middleware Building Blocks for Scalable Science

Balasubramanian,

Jha,

Merzky

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

RADICAL-Cybertools (RCT) are a set of software systems that serve as middleware to develop efficient and effective tools for scientific computing. Specifically, RCT enable executing many-task applications at extreme scale and on a variety of computing infrastructures. RCT are building blocks, designed to work as stand-alone systems, integrated among themselves or integrated with third-party systems. RCT enables innovative science in multiple domains, including but not limited to biophysics, climate science and particle physics, consuming hundreds of millions of core hours. This paper provides an overview of RCT components, their impact, and the architectural principle and software engineering underlying RCT.

show abstract

“…RCT eschew the concept of a monolithic workflow systems and uses "building blocks". RCT provide scalable implementations of building blocks in Python that are used to support dozens of scientific applications on high-performance and distributed systems [26], [3], [25], [7], [6]. In this Section we discuss details of RP, EnTK and HTBAC, understanding how these components have been used to support the flexible and scalable execution of pipelines.…”

Section: Radical-cybertoolsmentioning

confidence: 99%

High-throughput binding affinity calculations at extreme scales

Dakka

Turilli

Wright³

et al. 2018

BMC Bioinformatics

Self Cite

View full text Add to dashboard Cite

BackgroundResistance to chemotherapy and molecularly targeted therapies is a major factor in limiting the effectiveness of cancer treatment. In many cases, resistance can be linked to genetic changes in target proteins, either pre-existing or evolutionarily selected during treatment. Key to overcoming this challenge is an understanding of the molecular determinants of drug binding. Using multi-stage pipelines of molecular simulations we can gain insights into the binding free energy and the residence time of a ligand, which can inform both stratified and personal treatment regimes and drug development. To support the scalable, adaptive and automated calculation of the binding free energy on high-performance computing resources, we introduce the High-throughput Binding Affinity Calculator (HTBAC). HTBAC uses a building block approach in order to attain both workflow flexibility and performance.ResultsWe demonstrate close to perfect weak scaling to hundreds of concurrent multi-stage binding affinity calculation pipelines. This permits a rapid time-to-solution that is essentially invariant of the calculation protocol, size of candidate ligands and number of ensemble simulations.ConclusionsAs such, HTBAC advances the state of the art of binding affinity calculations and protocols. HTBAC provides the platform to enable scientists to study a wide range of cancer drugs and candidate ligands in order to support personalized clinical decision making based on genome sequencing and drug discovery.

show abstract

Evaluating Distributed Execution of Workloads

Cited by 11 publications

References 27 publications

Middleware Building Blocks for Workflow Systems

Middleware Building Blocks for Workflow Systems

RADICAL-Cybertools: Middleware Building Blocks for Scalable Science

High-throughput binding affinity calculations at extreme scales

Contact Info

Product

Resources

About