Using Kerberos Tokens in Distributed Computing System at IHEP

Abstract

The token-based certification method is spreading in the distributed computing system of high energy physics. More and more software and middleware are supporting tokens as one of the certification methods. As an example, WLCG has upgraded all the services to support WLCG tokens [1]. In IHEP (Institute of High Energy Physics in China), the Kerberos [2] token has been used as the main certification method in the local cluster. Naturally, it is selected as the certification method in the distributed computing system. In this case, a set of toolkits were developed or introduced to use Kerberos tokens in the distributed computing system, including token producer, token repository, token transfer and token client engine. The token producer is responsible for creating a token and publishing the token file to the token repository. The token repository stores all the latest token files and a refresh service periodically renews the lifetime of those tokens stored in the token repository. The token transfer brings the token file to the worker node. The token client engine initializes the token environment and renews the token’s lifetime on the worker node. With these toolkits, the jobs can run in any worker node in any site and use the Kerberos token to access other services, such as EOS [3] and the XRootd [4] proxy service. In IHEP, the Kerberos toolkit has been deployed in the distributed computing system. Currently, three experiments (LHAASO [5], BES [6] and HERD [7]) are using Kerberos tokens to remotely access the data in EOS or Lustre [8].