Evolution of grid-wide access to database resident information in ATLAS using Frontier

Abstract

The ATLAS experiment deployed Frontier technology worldwide during the initial year of LHC collision data taking to enable user analysis jobs running on the Worldwide LHC Computing Grid to access database resident data. Since that time, the deployment model has evolved to optimize resources, improve performance, and streamline maintenance of Frontier and related infrastructure. In this presentation we focus on the specific changes in the deployment and improvements undertaken such as the optimization of cache and launchpad location, the use of RPMs for more uniform deployment of underlying Frontier related components, improvements in monitoring, optimization of fail-over, and an increasing use of a centrally managed database containing site specific information (for configuration of services and monitoring). In addition, analysis of Frontier logs has allowed us a deeper understanding of problematic queries and understanding of use cases. Use of the system has grown beyond just user analysis and subsystem specific tasks such as calibration and alignment, extending into production processing areas such as initial reconstruction and trigger reprocessing. With a more robust and tuned system, we are better equipped to satisfy the still growing number of diverse clients and the demands of increasingly sophisticated processing and analysis.

Evaluation of the ATLAS model for remote access to database resident information for LHC Run 3

“…The problem was found to be proportional to distance along the WAN (wide area network) compounded by Oracle communication protocols needing back and forth communication between client and server beyond a simple request-response interaction. During the same period, we had already been in the process of evaluating Frontier technology [6] which is a caching system which distributes data from data sources (in our case, the Oracle databases) to distributed clients (in our case, tasks on the grid) using straight-forward HTTP protocols. Using a prototype Frontier deployment at one of the Tier 1 sites, our colleagues in Tokyo ran dedicated tests on their local grid site comparing database access times using the three available access methods: A graphical representation of the database access times observed from event processing tasks on local Tokyo worker nodes to three database sources: a local SQLite file, a Frontier deployment in New York (USA), and direct database access in Lyon ( France).…”

The Present and Future Challenges of Distributed Computing in the ATLAS experiment

“…Among the other changes made in the model, introducing the Frontier/Squid has enabled remote access to the databases at Tier-0 and Tier-1 centres from any site [8]. It is now possible, for example, to run reprocessing jobs that require detector conditions data at Tier-2 centres.…”

The challenges of mining logging data in ATLAS

“…A key component for grid-wide processing of ATLAS [1] event data is the global distribution of information from ATLAS databases which is necessary for that processing. Since the deployment of the Frontier technology [2] at grid sites early in Run 1, clients anywhere on the WLCG have benefited from efficient access to database-resident data, enabling a wide variety of event processing and analysis.…”