The Atlas Eventindex Using the Hbase/Phoenix Storage Solution

Cherepanova, Elizaveta; Alexandrov, Evgeny; Aleksandrov, I. N.; Barberis, D.; Canali, Luca; Casaní, Álvaro Fernández; Montoro, Carlos García; Hoz, S. González de la; Hřivnáć, J.; Kazymov, Andrei; Mineev, M.; Prokoshin, F.; Rybkin, G; Sánchez, Juan Enrique Prudencio; Cairols, José Salt; Perez, M. Villaplana; Yakovlev, A.

doi:10.54546/mlit.2021.68.25.001

9th International Conference "Distributed Computing and Grid Technologies in Science and Education" 2021

DOI: 10.54546/mlit.2021.68.25.001

|View full text |Cite

The Atlas Eventindex Using the Hbase/Phoenix Storage Solution

Elizaveta Cherepanova¹,

Evgeny Alexandrov²,

I. N. Aleksandrov³

et al.

Abstract: The ATLAS EventIndex provides a global event catalogue and event-level metadata for ATLAS analysis groups and users. The LHC Run 3, starting in 2022, will see increased data-taking and simulation production rates, with which the current infrastructure would still cope but may be stretched to its limits by the end of Run 3. This talk describes the implementation of a new core storage service that will provide at least the same functionality as the current one for increased data ingestion and search rates, and w… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

The ATLAS EventIndex

Barberis

Aleksandrov²,

Alexandrov³

et al. 2023

Comput Softw Big Sci

View full text Add to dashboard Cite

The ATLAS EventIndex system comprises the catalogue of all events collected, processed or generated by the ATLAS experiment at the CERN LHC accelerator, and all associated software tools to collect, store and query this information. ATLAS records several billion particle interactions every year of operation, processes them for analysis and generates even larger simulated data samples; a global catalogue is needed to keep track of the location of each event record and be able to search and retrieve specific events for in-depth investigations. Each EventIndex record includes summary information on the event itself and the pointers to the files containing the full event. Most components of the EventIndex system are implemented using BigData free and open-source software. This paper describes the architectural choices and their evolution in time, as well as the past, current and foreseen future implementations of all EventIndex components.

show abstract

The ATLAS EventIndex

Barberis

Aleksandrov²,

Alexandrov³

et al. 2023

Comput Softw Big Sci

View full text Add to dashboard Cite

show abstract

Big Data Analytics for the ATLAS EventIndex Project with Apache Spark

Fernández Casaní,

García Montoro,

González de la Hoz

et al. 2023

Computational and Mathematical Methods

View full text Add to dashboard Cite

The ATLAS EventIndex was designed to provide a global event catalogue and limited event-level metadata for ATLAS experiment of the Large Hadron Collider (LHC) and their analysis groups and users during Run 2 (2015-2018) and has been running in production since. The LHC Run 3, started in 2022, has seen increased data-taking and simulation production rates, with which the current infrastructure would still cope but may be stretched to its limits by the end of Run 3. A new core storage service is being developed in HBase/Phoenix, and there is work in progress to provide at least the same functionality as the current one for increased data ingestion and search rates and with increasing volumes of stored data. In addition, new tools are being developed for solving the needed access cases within the new storage. This paper describes a new tool using Spark and implemented in Scala for accessing the big data quantities of the EventIndex project stored in HBase/Phoenix. With this tool, we can offer data discovery capabilities at different granularities, providing Spark Dataframes that can be used or refined within the same framework. Data analytic cases of the EventIndex project are implemented, like the search for duplicates of events from the same or different datasets. An algorithm and implementation for the calculation of overlap matrices of events across different datasets are presented. Our approach can be used by other higher-level tools and users, to ease access to the data in a performant and standard way using Spark abstractions. The provided tools decouple data access from the actual data schema, which makes it convenient to hide complexity and possible changes on the backed storage.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

The Atlas Eventindex Using the Hbase/Phoenix Storage Solution

Cited by 2 publications

References 7 publications

The ATLAS EventIndex

The ATLAS EventIndex

Big Data Analytics for the ATLAS EventIndex Project with Apache Spark

Contact Info

Product

Resources

About