Towards a responsive CernVM-FS architecture

Abstract

The CernVM File System (CernVM-FS) provides a scalable and reliable software distribution service implemented as a POSIX read-only filesystem in user space (FUSE). It was originally developed at CERN to assist High Energy Physics (HEP) collaborations in deploying software on the worldwide distributed computing infrastructure for data processing applications. Files are stored remotely as content-addressed blocks on standard web servers and are retrieved and cached on-demand through outgoing HTTP connections only. Repository metadata is recorded in SQLite catalogs, which represent implicit Merkle treeencodings of the repository state. For writing, CernVM-FS follows a publish-subscribe pattern with a single source of new content that is propagated to a large number of readers. This paper focuses on the work to move the CernVM-FS architecturein the direction of a responsive data distribution system. A new distributed publication backend allows scaling out large publication tasks across multiple machines, reducing the time to publish. For the faster propagation of new published content, the addition of a notification system allows clients to subscribe to messages about changes in the repository and to request new root catalogs as soon as they become available. These devel-opments make CernVM-FS more responsive and are particularly relevant for use cases where a short propagation delay from repository down to individual clients is important, such as using CernVM-FS as an AFS replacement for distributing software stacks. Additionally, they permit the implementation of more complex workflows, with producer-consumer pipelines, as for example in the ALICE analysis trains system.

CernVM-FS ephemeral publishers on Kubernetes

“…Since the publishing via the release manager machine allows to modify the repository content only from one single publisher node at a time, each CernVM-FS repository used to have one single and carefully maintained publisher node. A gateway component was added to the CernVM-FS publishing tools as of version 2.5 in order to enable multiple publisher nodes that operate on the same repository concurrently [6]. The gateway consists of a web service in front of the authoritative storage, so that it is the only entity able to write into the storage of the managed repositories.…”

A Subset of the CERN Virtual Machine File System: Fast Delivering of Complex Software Stacks for Supercomputing Resources

“…Changes are written into a staging area until they are committed as a consistent changeset: new and modified files are transformed into a content-addressed object providing file-based deduplication and versioning. In 2019, Popescu et al [43] introduced a gateway component, a web service in front of the authoritative storage, allowing release managers to perform concurrent operations on the same repository and make CVMFS more responsive (Figure 1.1.b and 1.2.b).…”

“…The transfer of files is then done lazily via HTTP connections initiated by the CVMFS clients [43] (Figure 1.3.b). Clients request updates based on their Time-to-Live (TTL) value, which is generally about a few minutes.…”

Towards a serverless CernVM-FS

“…Current efforts are focused on reducing the content propagation from the around 10 minutes default to less than one minute. [8] 3. Unpacked Container Images.…”

“…As of CernVM-FS 2.5, a gateway component has been added to the CernVM-FS publishing tools [8]. The gateway is used in production, for instance, for the cernvm-prod.cern.ch repository.…”