Evolution of the CMS Global Submission Infrastructure for the HL-LHC Era

Yzquierdo, A. Pérez-Calero; Flechas, Maria Acosta; Foyo, Diego Davila; Haleem, Saqib; Anampa, K. Hurtado; Ivanov, T.; Khan, Farrukh Aftab; Kizinevič, Edita; Larson, Krista; Letts, J.; Mascheroni, Marco; Mason, D.

doi:10.1051/epjconf/202024503016

Cited by 6 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SI comprises multiple interconnected HTCondor pools [372], as shown in figure 139, redundantly deployed at CERN and FNAL in order to ensure a high-availability service. The main component of the SI, the Global Pool, obtains the majority of its resources via the submission of pilot jobs to WLCG [328] and open science grid (OSG) sites.…”

Section: Central Processing and Productionmentioning

confidence: 99%

See 1 more Smart Citation

Development of the CMS detector for the CERN LHC Run 3

Hayrapetyan,

Tumasyan,

Adam

et al. 2024

J. Inst.

View full text Add to dashboard Cite

Since the initial data taking of the CERN LHC, the CMS experiment has undergone substantial upgrades and improvements. This paper discusses the CMS detector as it is configured for the third data-taking period of the CERN LHC, Run 3, which started in 2022. The entire silicon pixel tracking detector was replaced. A new powering system for the superconducting solenoid was installed. The electronics of the hadron calorimeter was upgraded. All the muon electronic systems were upgraded, and new muon detector stations were added, including a gas electron multiplier detector. The precision proton spectrometer was upgraded. The dedicated luminosity detectors and the beam loss monitor were refurbished. Substantial improvements to the trigger, data acquisition, software, and computing systems were also implemented, including a new hybrid CPU/GPU farm for the high-level trigger.

show abstract

Section: Central Processing and Productionmentioning

confidence: 99%

“…Schematic diagram of the submission infrastructure, including multiple distributed central processing and production (WMAgent) and analysis (CRAB) job submission agents (schedds). Reproduced from[372]. CC BY 4.0.…”

mentioning

confidence: 99%

Development of the CMS detector for the CERN LHC Run 3

Hayrapetyan,

Tumasyan,

Adam

et al. 2024

J. Inst.

View full text Add to dashboard Cite

show abstract

“…Figure 1 presents a schematic view of this infrastructure. The SI is in continuous evolution (see for example [8]), managing an ever growing collection of computing resources, connecting new and more diverse resource types and resource providers (WLCG, HPC, Cloud, volunteer). The main challenge for the SI team is to drive the evolution of our infrastructure, while maintaining the efficiency of use in all the available resources, maximizing data processing throughput and enforcing task priorities according to CMS research program.…”

Section: The Cms Submission Infrastructurementioning

confidence: 99%

“…Test jobs are submitted to the SI schedds targeting any available GPUs, with the intention of matching as many as possible. Allocation of GPUs peaked at over 150 GPUs in parallel, accessing in total about 230 unique opportunistic GPUs, located mainly at CERN and US Tier-2 sites (see [15]).…”

Section: Availability Of Gpus In Cms Simentioning

confidence: 99%

The integration of heterogeneous resources in the CMS Submission Infrastructure for the LHC Run 3 and beyond

Pérez-Calero Yzquierdo,

Mascheroni,

Kizinevic

et al. 2024

EPJ Web of Conf.

Self Cite

View full text Add to dashboard Cite

While the computing landscape supporting LHC experiments is currently dominated by x86 processors at WLCG sites, this configuration will evolve in the coming years. LHC collaborations will be increasingly employing HPC and Cloud facilities to process the vast amounts of data expected during the LHC Run 3 and the future HL-LHC phase. These facilities often feature diverse compute resources, including alternative CPU architectures like ARM and IBM Power, as well as a variety of GPU specifications. Using these heterogeneous resources efficiently is thus essential for the LHC collaborations reaching their future scientific goals. The Submission Infrastructure (SI) is a central element in CMS Computing, enabling resource acquisition and exploitation by CMS data processing, simulation and analysis tasks. The SI must therefore be adapted to ensure access and optimal utilization of this heterogeneous compute capacity. Some steps in this evolution have been already taken, as CMS is currently using opportunistically a small pool of GPU slots provided mainly at the CMS WLCG sites. Additionally, Power9 processors have been validated for CMS production at the Marconi-100 cluster at CINECA. This note will describe the updated capabilities of the SI to continue ensuring the efficient allocation and use of computing resources by CMS, despite their increasing diversity. The next steps towards a full integration and support of heterogeneous resources according to CMS needs will also be reported.

show abstract

“…not involving for example experimental data reprocessing tasks). Technically, such restrictions would be enforced by using a similar approach to that of CMS site-customizable pilots [17]. For the current tests, only payload jobs known to work a priori (described in the next section) were allowed matchmaking the BSC slots.…”

Section: Integration With Cms Workload Management Systemsmentioning

confidence: 99%

Exploitation of network-segregated CPU resources in CMS

et al. 2021

View full text Add to dashboard Cite

CMS is tackling the exploitation of CPU resources at HPC centers where compute nodes do not have network connectivity to the Internet. Pilot agents and payload jobs need to interact with external services from the compute nodes: access to the application software (CernVM-FS) and conditions data (Frontier), management of input and output data files (data management services), and job management (HTCondor). Finding an alternative route to these services is challenging. Seamless integration in the CMS production system without causing any operational overhead is a key goal. The case of the Barcelona Supercomputing Center (BSC), in Spain, is particularly challenging, due to its especially restrictive network setup. We describe in this paper the solutions developed within CMS to overcome these restrictions, and integrate this resource in production. Singularity containers with application software releases are built and pre-placed in the HPC facility shared file system, together with conditions data files. HTCondor has been extended to relay communications between running pilot jobs and HTCondor daemons through the HPC shared file system. This operation mode also allows piping input and output data files through the HPC file system. Results, issues encountered during the integration process, and remaining concerns are discussed.

show abstract

Evolution of the CMS Global Submission Infrastructure for the HL-LHC Era

Cited by 6 publications

References 7 publications

Development of the CMS detector for the CERN LHC Run 3

Development of the CMS detector for the CERN LHC Run 3

The integration of heterogeneous resources in the CMS Submission Infrastructure for the LHC Run 3 and beyond

Exploitation of network-segregated CPU resources in CMS

Contact Info

Product

Resources

About