Xcache in the ATLAS Distributed Computing Environment

Hanushevsky, Andrew; Ito, H.; Lassnig, M.; Popescu, Radu; Silva, Asoka De; Simon, M.; Garonne, V.; Stefano, J De; Vukotić, I.; Yang, Wei

doi:10.1051/epjconf/201921404008

Cited by 8 publications

(6 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A longer campaign of measurements is foreseen to get more precise measurements. To optimise the network occupancy and further minimise the impact of latency, tools like Xcache [16] will be included and new measurements will be done.…”

Section: Resultsmentioning

confidence: 99%

Implementation and performance of a DPM federated storage and integration within the ATLAS environment

et al. 2020

View full text Add to dashboard Cite

With the increase of storage needs at the High-Luminosity LHC horizon, data management and access will be very challenging. The evaluation of possible solutions within the WLCG Data Organization, Management and Access (DOMA) is a major activity to select the most optimal from the experiment and site point of views. Four teams hosting Tier-2s for ATLAS with storage based on DPM technology have put their expertise and computing infrastructures in common to build a testbed hosting a DPM federated storage called FR-ALPES. This note describes the infrastructure put in place, its integration within the ATLAS Grid infrastructure and presents the first results.

show abstract

Section: Resultsmentioning

confidence: 99%

Implementation and performance of a DPM federated storage and integration within the ATLAS environment

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The Kerberos token is stored in the job executing directory and initialized into the job environment when the job arrived at the remote worker node. With the Kerberos token, the job can access storage [10] and EOS in the IHEP case (the figure below shows the use of Kerberos token to access Lustre and XCache). As the certification of Kerberos tokens cover the local computing cluster and the remote sites, it can be potentially expanded to other kinds of services in the future, not only the storage service.…”

Section: Kerberos Tokens In Distributed Computing Systemmentioning

confidence: 99%

Using Kerberos Tokens in Distributed Computing System at IHEP

Jiang,

Guo,

et al. 2024

EPJ Web of Conf.

View full text Add to dashboard Cite

The token-based certification method is spreading in the distributed computing system of high energy physics. More and more software and middleware are supporting tokens as one of the certification methods. As an example, WLCG has upgraded all the services to support WLCG tokens [1]. In IHEP (Institute of High Energy Physics in China), the Kerberos [2] token has been used as the main certification method in the local cluster. Naturally, it is selected as the certification method in the distributed computing system. In this case, a set of toolkits were developed or introduced to use Kerberos tokens in the distributed computing system, including token producer, token repository, token transfer and token client engine. The token producer is responsible for creating a token and publishing the token file to the token repository. The token repository stores all the latest token files and a refresh service periodically renews the lifetime of those tokens stored in the token repository. The token transfer brings the token file to the worker node. The token client engine initializes the token environment and renews the token’s lifetime on the worker node. With these toolkits, the jobs can run in any worker node in any site and use the Kerberos token to access other services, such as EOS [3] and the XRootd [4] proxy service. In IHEP, the Kerberos toolkit has been deployed in the distributed computing system. Currently, three experiments (LHAASO [5], BES [6] and HERD [7]) are using Kerberos tokens to remotely access the data in EOS or Lustre [8].

show abstract

“…For that reason, it is often used at lightweight sites to replace a distributed file system. Recent developments like XCache [75] also offer transparent caching.…”

Section: Xrootdmentioning

confidence: 99%

Operating an HPC/HTC Cluster with Fully Containerized Jobs Using HTCondor, Singularity, CephFS and CVMFS

Freyermuth

Wienemann

Bechtle

et al. 2021

Comput Softw Big Sci

View full text Add to dashboard Cite

High performance and high throughput computing (HPC/HTC) is challenged by ever increasing demands on the software stacks and more and more diverging requirements by different research communities. This led to a reassessment of the operational concept of HPC/HTC clusters at the Physikalisches Institut at the University of Bonn. As a result, the present HPC/HTC cluster (named BAF2) introduced various conceptual changes compared to conventional clusters. All jobs are now run in containers and a container-aware resource management system is used which allowed us to switch to a model without login/head nodes. Furthermore, a modern, feature-rich storage system with powerful interfaces has been deployed. We describe the design considerations, the implemented functionality and the operational experience gained with this new-generation setup which turned out to be very successful and well-accepted by its users.

show abstract

Xcache in the ATLAS Distributed Computing Environment

Cited by 8 publications

References 4 publications

Implementation and performance of a DPM federated storage and integration within the ATLAS environment

Implementation and performance of a DPM federated storage and integration within the ATLAS environment

Using Kerberos Tokens in Distributed Computing System at IHEP

Operating an HPC/HTC Cluster with Fully Containerized Jobs Using HTCondor, Singularity, CephFS and CVMFS

Contact Info

Product

Resources

About