LHC Data Storage: Preparing for the Challenges of Run-3

Abstract

The CERN IT Storage Group ensures the symbiotic development and operations of storage and data transfer services for all CERN physics data, in particular the data generated by the four LHC experiments (ALICE, ATLAS, CMS and LHCb). In order to accomplish the objectives of the next run of the LHC (Run-3), the Storage Group has undertaken a thorough analysis of the experiments’ requirements, matching them to the appropriate storage and data transfer solutions, and undergoing a rigorous programme of testing to identify and solve any issues before the start of Run-3. In this paper, we present the main challenges presented by each of the four LHC experiments. We describe their workflows, in particular how they communicate with and use the key components provided by the Storage Group: the EOS disk storage system; its archival back-end, the CERN Tape Archive (CTA); and the File Transfer Service (FTS). We also describe the validation and commissioning tests that have been undertaken and challenges overcome: the ATLAS stress tests to push their DAQ system to its limits; the CMS migration from PhEDEx to Rucio, followed by large-scale tests between EOS and CTA with the new FTS “archive monitoring” feature; the LHCb Tier-0 to Tier-1 staging tests and XRootD Third Party Copy (TPC) validation; and the erasure coding performance in ALICE.

XRootD Client: A robust technology for LHC Run-3 and beyond

“…The challenges for Run-3 of the LHC [1], which began in 2022, include managing the massive amounts of data generated by the LHC experiments and ensuring their efficient storage, access, and analysis. In order to address these challenges, XRootD [2,3] is now widely used by the High Energy Physics (HEP) community.…”

Distributed Computing Solutions for High Energy Physics Interactive Data Analysis

“…Consequently, the volumes of data that are continuously accessed by researchers reach even higher limits. For example, the CERN storage service, EOS, has served 2.5 Exabytes (EB) of data to users only in the year 2020 [21]. This trend will continue in the current Run 3 and future runs as well.…”