The ATLAS EventIndex for LHC Run 3

Self Cite

The ATLAS EventIndex is the global catalogue of all ATLAS real and simulated events. During the LHC long shutdown between Run 2 (20152018) and Run 3 (2022-2025) its components were substantially revised, and a new system was deployed for the start of Run 3 in Spring 2022. The new core storage system is based on HBase tables with a Phoenix interface. It allows faster data ingestion rates and scales better than the old system. This paper describes the data collection, the technical design of the core storage, and the properties that make it fast and efficient, namely the compact and optimized design of the events table, which already holds more than 400 billion entries, and all the auxiliary tables, and the EventIndex Supervisor, in charge of orchestrating the whole data collection, now simplified thanks to the Loaders, the Spark jobs that load the data into the new core system. The extractors, in charge of preparing the pieces of data that the loaders will put into the final back-end, have been updated too. The data migration from HDFS to HBase and Phoenix is also described.

Section: Data Storage Logical Architecturementioning

confidence: 99%

HBase/Phoenix-based Data Collection and Storage for the ATLAS EventIndex

García Montoro,

Sánchez,

Barberis

et al. 2024

Self Cite

“…These tools evolve quite rapidly in time, as additional features become available and their performance improves; any system using them must also evolve in parallel. All EventIndex components were revised or reimplemented in advance of the start of LHC Run 3 in 2022 [16].…”

Section: Revised and New Componentsmentioning

confidence: 99%

“…The original version based on Hadoop MapFiles was developed in 2014 and soon complemented by an additional data store implemented in HBase in order to meet the requirements on query response time. The current version is based on HBase tables, with Phoenix as the SQL interface [16,19]. As all events in ATLAS belong to datasets (groups of files containing statistically similar events), it was logical to organise the data into a Dataset table and an Events table; the Events table is very large, with over 500 billion event records in Spring 2023, but with data grouped into column families and an optimized primary key that ensures a balanced use of all HBase regions.…”

Section: Data Storementioning

confidence: 99%

Deployment and Operation of the ATLAS EventIndex for LHC Run 3

Gallas,

Alexandrov,

Alexandrov

et al. 2024

Self Cite

The ATLAS Eventlndex is the global catalogue of all ATLAS real and simulated events. During the LHC long shutdown between Run 2 (20152018) and Run 3 (2022-2025) all its components were substantially revised and a new system was deployed for the start of Run 3 in Spring 2022. The new core storage system, based on HBase tables with a SQL interface provided by Phoenix, allows much faster data ingestion rates and scales much better than the old one to the data rates expected for the end of Run 3 and beyond. All user interfaces were also revised and a new command-line interface and web services were also deployed. The new system was initially populated with all existing data relative to Run 1 and Run 2 datasets, and then put online to receive Run 3 data in real time. After extensive testing, the old system, which ran in parallel to the new one for a few months, was finally switched off in October 2022. This paper describes the new system, the move of all existing data from the old to the new storage schemas and the operational experience gathered so far.

“…A schema of the EventIndex architecture implemented for LHC Run 3 is shown in Figure 1 and a description of every component can be found in Ref. [4]. Ideally, the system should be available and accessible under various conditions and give a response in the appropriate time, depending on the request.…”

Section: Introductionmentioning

confidence: 99%

Testing framework and monitoring system for the ATLAS EventIndex

Cherepanova,

Gallas,

Prokoshin

et al. 2024

The ATLAS EventIndex is a global catalogue of the events collected, processed or generated by the ATLAS experiment. The system was upgraded in advance of LHC Run 3, with a migration of the Run 1 and Run 2 data from HDFS MapFiles to HBase tables with a Phoenix interface. Two frameworks for testing functionality and performance of the new system have been developed. There are two types of tests running. First, the functional test that must check the correct functioning of the import chain. These tests run event picking over a random set of recently imported data to see if the data have been imported correctly, and can be accessed by both the CLI and the PanDA client. The second, the performance test, generates event lookup queries on sets of the EventIndex data and measures the response times. These tests enable studies of the response time dependence on the amount of requested data, and data sample type and size. Both types of tests run regularly on the existing system. The results of the regular tests as well as the statuses of the main EventIndex subsystems (services health, loaders status, filesystem usage, etc.) are sent to InfluxDB in JSON format via HTTP requests and are displayed on Grafana monitoring dashboards. In case (part of) the system misbehaves or becomes unresponsive, alarms are raised by the monitoring system.