The CernVM File System (CernVM-FS) provides a scalable and reliable software distribution and-to some extent-a data distribution service. It gives POSIX access to more than a billion binary files of experiment application software stacks and operating system containers to end user devices, grids, clouds, and supercomputers. Increasingly, CernVM-FS also provides access to certain classes of data, such as detector conditions data, genomics reference sets, or gravitational wave detector experiment data. For most of the highenergy physics experiments, an underlying HTTP content distribution infrastructure is jointly provided by universities and research institutes around the world. In this contribution, we will present recent developments and future plans. For future developments, we put a focus on evolving the content distribution infrastructure and at lowering the barrier for publishing into CernVM-FS. Through so-called serverless computing, we envision cloud hosted CernVM-FS repositories without the need to operate dedicated servers or virtual machines. An S3 compatible service in conjunction with a content delivery network takes on data provisioning, replication, and caching. A chain of time-limited and resource-limited functions (so called "lambda function" or "function-asa-service") operate on the repository and stage the updates. As a result, any CernVM-FS client should be able to turn into a writer, possession of suitable keys provided. For repository owners, we aim at providing cost transparency and seamless scalability from very small to very large
The CernVM File System provides the software and container distribution backbone for most High Energy and Nuclear Physics experiments. It is implemented as a file system in user-space (Fuse) module, which permits its execution without any elevated privileges. Yet, mounting the file system in the first place is handled by a privileged suid helper program that is installed by the Fuse package on most systems. The privileged nature of the mount system call is a serious hindrance to running CernVM-FS on opportunistic resource and supercomputers. Fortunately, recent developments in the Linux kernel and in the Fuse user-space libraries enabled fully unprivileged mounting for Fuse file systems (as of RHEL 8), or at least outsourcing the privileged mount system call to a custom, external process. This opens the door to several, very appealing new ways to use CernVM-FS, such as a generally usable “super pilot” consisting of the pilot code bundled with Singularity and CernVM-FS, or the on-demand instantiation of unprivileged, ephemeral containers to publish new CernVM-FS content from anywhere. In this contribution, we discuss the integration of these new Linux features with CernVM-FS and show some of its most promising, new applications.
Containers became the de-facto standard to package and distribute modern applications and their dependencies. The HEP community demonstrates an increasing interest in such technology, with scientists encapsulating their analysis workflow and code inside a container image. The analysis is first validated on a small dataset and minimal hardware resources to then run at scale on the massive computing capacity provided by the grid. The typical approach for distributing containers consists of pulling their image from a remote registry and extracting it on the node where the container runtime (e.g., Docker, Singularity) runs. This approach, however, does not easily scale to large images and thousands of nodes. CVMFS has long been used for the efficient distribution of software directory trees at a global scale. In order to extend its optimized caching and network utilization to the distribution of containers, CVMFS recently implemented a dedicated container image ingestion service together with container runtime integrations. CVMFS ingestion is based on per-file deduplication, instead of the per-layer deduplication adopted by traditional container registries. On the client-side, CVMFS implements on-demand fetching of the chunks required for the execution of the container instead of the whole image.
The past years have shown a revolution in the way scientific workloads are being executed thanks to the wide adoption of software containers. These containers run largely isolated from the host system, ensuring that the development and execution environments are the same everywhere. This enables full reproducibility of the workloads and therefore also the associated scientific analyses performed. However, as the research software used becomes increasingly complex, the software images grow easily to sizes of multiple gigabytes. Downloading the full image onto every single compute node on which the containers are executed becomes unpractical. In this paper, we describe a novel way of distributing software images on the Kubernetes platform, with which the container can start before the entire image contents become available locally (so-called “lazy pulling”). Each file required for the execution is fetched individually and subsequently cached on-demand using the CernVM file system (CVMFS), enabling the execution of very large software images on potentially thousands of Kubernetes nodes with very little overhead. We present several performance benchmarks making use of typical high-energy physics analysis workloads.
Linux containers have gained widespread use in high energy physics, be it for services using container engines such as containerd/kubernetes, for production jobs using container engines such as Singularity or Shifter, or for development workflows using Docker as a local container engine. Thus the efficient distribution of the container images, whose size usually ranges from a few hundred megabytes to a few tens of gigabytes, is becoming a pressing concern. Because container images show similar characteristics than scientific application stacks, unpacking the images in CernVM-FS can remedy the distribution issues provided that the container engine at hand is able to use such unpacked images from CernVM-FS. In this contribution, we willl report on recent advances in the integration of Singularity, Docker, and containerd with CernVM-FS. We show improvements in the publishing of container images from a Docker registry that rely on new means of directly ingesting image tarballs. Well also show a repository file system structure for storing container images that are optimized for storing both container engines using flat root file systems (Singularity) as well as container engines using layers (containerd, Docker). To evaluate the benefits of our approach, we show concrete use cases and figures for production and development images from LHC experiments stored in the recently created unpacked.cern.ch repository.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.