Evaluation of fault-tolerant policies using simulation

Abstract

Abstract-Various mechanisms for fault-tolerance (FT) are used today in order to reduce the impact of failures on application execution. In the case of system failure, standard FT mechanisms are checkpoint/restart (for reactive FT) and migration (for pro-active FT). However, each of these mechanisms create an overhead on application execution, overhead that for instance becomes critical on large-scale systems where previous studies have shown that applications may spend more time checkpointing state than performing useful work.In order to decrease this overhead, researchers try to both optimize existing FT mechanisms and implement new FT policies. For instance, combining reactive and pro-active approaches in order to decrease the number of checkpoints that must be performed during the application's execution. However, currently no solutions exist which enable the evaluation of these FT approaches through simulation, instead experimentations must be done using real platforms. This increases complexity and limits experimentation into alternate solutions. This paper presents a simulation framework that evaluates different FT mechanisms and policies. The framework uses system failure logs for the simulation with a default behavior based on logs taken from the ASCI White at Lawrence Livermore National Laboratory. We evaluate the accuracy of our simulator comparing simulated results with those taken from experiments done on a 32-node compute cluster. Therefore such a simulator can be used to develop new FT policies and/or to tune existing policies.

A Framework for Proactive Fault Tolerance

“…Instead we decided to compare the results of the implementation of a specific proactive fault tolerance policy with the results of the same policy obtained by simulation. Our simulator [8] is based on the LLNL's ASCI white system system logs and our experimentation platform consists of a 40 nodes cluster. Each physical node has a single Xen VM having 250MB of memory (the number of VM is explicitly specified if VMs are stacked on physical machines); host OSes have 200MB of memory.…”

“…However, the architecture is designed to be easily extended to other mechanisms, such as process migration and process pause/unpause. This framework, coupled to our fault tolerance simulator [8] provides a complete set of tools for the study of proactive fault tolerance policies.…”

Proactive process-level live migration and back migration in HPC environments

“…They also model the migration cost and introduce a dynamic scheduling mechanism accordingly [14]. In their paper, Tikotekar et al also present a simulation framework that evaluates different FT mechanisms and policies, including a combination of reactive FT and proactive FT to decrease the number of checkpoints [48], which obtained the best results among all the real and simulated FT mechanisms and policies. These prior works with their fault models, FT mechanisms for fault occurrences and their evaluation simulations, confirm that the process migration is a suitable approach for proactive FT with lower cost than OS virtualization, which reinforces the significance of our solution.…”

“…In this spirit, our work focuses on fault-tolerant middleware for HPC systems. More specifically, this paper promotes process-level live migration combined with health monitoring for a proactive FT approach that complements existing C/R schemes with self healing whose fault model is based on the work by Tikotekar et al [48].…”

Proactive Fault Tolerance Using Preemptive Migration

“…Recent work in this area utilized simulations to evaluate different trade-off models when combining preemptive migration with checkpoint/restart [11]. Using failure logs from Lawrence Livermore National Laboratory, the impact of failure prediction accuracy was evaluated and put in context with restart counts and checkpointing frequency.…”