A Synchronization-Induced Checkpoint Protocol for Group-Synchronous Parallel Programs

Abstract

Group checkpointing is a fix between global checkpointing and log-based recovery. It features both reduced runtime overhead and localized recovery effect for improving the fault-tolerance performance of large-scale distributed systems. However, parallel programs cannot efficiently benefit from this strategy, as they often involve synchronous or semisynchronous interactions that incur extra idling delays between processes as well as between process groups. This paper presents an analytical study on such delays and the corresponding delay optimization strategies. Observing that certain parallel programs exhibit patterns of "synchronization groups", we develop a Synchronization-Induced Checkpoint protocol that manages checkpoints around such groups. The protocol keeps advantages of ordinary group checkpointing, and meanwhile minimizes the costs of synchronization-induced delays. We also broadly categorize the known synchronization patterns and establish their relations with suitable checkpoint strategies for parallel programs. (Abstract)