Checkpointing Process Groups in a Grid Environment

Abstract

International audienceThe EU-funded XtreemOS project implements a grid operating system transparently exploiting resources of virtual organizations through the standard POSIX interface. Grid checkpointing and restart requires to save and restore jobs executing in a distributed heterogeneous grid environment. The latter may spawn millions of grid nodes ( PCs, clusters, and mobile devices ) using different system-specific checkpointers saving and restoring application and kernel data structures for processes executing on a grid node. In this paper we shortly describe the XtreemOS grid checkpointing architecture and how we bridge the gap between the abstract grid and the system-specific checkpointers. Then we discuss how we keep track of processes and how different process grouping techniques are managed to ensure that all processes of a job and any further dependent ones can be checkpointed and restarted. Finally, we present how Linux control groups can be used to address resource isolation issues during the restart

The Architecture of the XtreemOS Grid Checkpointing Service

“…Since no separate UNIX session or UNIX process group is initiated at job submission, BLCR cannot use these two process group semantics. More information can be found under [18].…”

“…The kernel is still able to distinguish equally named identifiers used across multiple applications, by isolating resource groups and mapping each identifier to a unique one at kernel level. Integration of state-of-the-art lightweight virtualization mechanisms provided by mainline Linux into XtreemOS is in progress, see also [18].…”

A Synchronization-Induced Checkpoint Protocol for Group-Synchronous Parallel Programs

“…Thus it provides a more flexible solution with advantages of both reduced runtime overhead and localized recovery effect. Recent reports have shown feasibility as well as efficiency of applying group checkpoints to certain MPI programs [13] and in specific grid computing environments [14].…”