HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System

Abstract

HEPCloud is rapidly becoming the primary system for provisioning compute resources for all Fermilab-affiliated experiments. In order to reliably meet the peak demands of the next generation of High Energy Physics experiments, Fermilab must plan to elastically expand its computational capabilities to cover the forecasted need. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and at which scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources spanning multiple cloud providers, multiple HPC centers, and grid computing federations. In this paper, we discuss the goals and architecture of the HEPCloud Facility, the architecture of the IDSS, and our early experience in using the IDSS for automated facility expansion both at Fermi and Brookhaven National Laboratory.

DUNE Offline Computing (Conceptual Design Report)

“…Figure 13.1 illustrates the entire chain, including interaction with storage elements within the job. The architecture can also provision resources on high-performance computing (HPC) resources, such as Cori at the NERSC, within the HEPCloud [141] infrastructure. The submission mechanism is unchanged whether the jobs are High Throughput Computing (HTC) or HPC; this seamless transition is key to efficiently utilizing available resources and also saves the job submitter significant effort by not requiring customized submission infrastructure for different resource types.…”

GPU coprocessors as a service for deep learning inference in high energy physics

“…FACILE is particularly well-suited for an online computing application because the algorithm it replaces is responsible for 15% of the HLT latency per event. In this study, the clients are deployed as jobs running single-thread HLT instances on virtual machines in Google Cloud using the HEPCloud framework [40][41][42]. HEPCloud deploys jobs submitted on batch systems to CPU instances created dynamically at a cloud computing site.…”

DUNE Production processing and workflow management software evaluation

“…Figure 1 illustrates the entire chain, including interaction with storage elements within the job. The architecture can also provision resources on High-Performance Computing (HPC) resources, such as Cori at the National Energy Research supercomputing Center (NERSC), within the HEP-Cloud [7,8] infrastructure. The submission mechanism is unchanged whether the jobs are HTC or HPC; this seamless transition is key to efficiently utilizing available resources and also saves the job submitter significant effort by not requiring customized submission infrastructure for different resource types.…”