Checkpoint-restart for a network of virtual machines

Garg, Rohan; Sodha, Komal; Jin, Zhengping; Cooperman, Gene

doi:10.1109/cluster.2013.6702626

Cited by 13 publications

(11 citation statements)

References 14 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(See Section 8.1 for a discussion of MPI checkpoint-restart services.) Kangarlou et al [KEX12] and Garg et al [GSJC13] each show how to take a distributed snapshot of VMs. Kangarlou et al base this on a modification of Xen's live migration, while Garg et al employ DMTCP to take a distributed snapshot of KVM/QEMU VMs.…”

Section: Mechanisms Based On Vm Snapshotsmentioning

confidence: 99%

Checkpointing as a Service in Heterogeneous Cloud Environments

Cao

Simonin

Cooperman³

et al. 2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

Self Cite

View full text Add to dashboard Cite

A non-invasive, cloud-agnostic approach is demonstrated for extending existing cloud platforms to include checkpoint-restart capability. Most cloud platforms currently rely on each application to provide its own fault tolerance. A uniform mechanism within the cloud itself serves two purposes: (a) direct support for long-running jobs, which would otherwise require a custom fault-tolerant mechanism for each application; and (b) the administrative capability to manage an over-subscribed cloud by temporarily swapping out jobs when higher priority jobs arrive. An advantage of this uniform approach is that it also supports parallel and distributed computations, over both TCP and InfiniBand, thus allowing traditional HPC applications to take advantage of an existing cloud infrastructure. Additionally, an integrated health-monitoring mechanism detects when longrunning jobs either fail or incur exceptionally low performance, perhaps due to resource starvation, and proactively suspends the job. The cloud-agnostic feature is demonstrated by applying the implementation to two very different cloud platforms: Snooze and OpenStack. The use of a cloud-agnostic architecture also enables, for the first time, migration of applications from one cloud platform to another.

show abstract

Section: Mechanisms Based On Vm Snapshotsmentioning

confidence: 99%

Checkpointing as a Service in Heterogeneous Cloud Environments

Cao

Simonin

Cooperman³

et al. 2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…The process migration indeed is very much used in bigdata business process. The customers in today's world relay in the digital content and the application process that supports and manages these need to be managed with better efficient and in convenient ways [3,4]. In [5,6], the process of checkpoint and restarting the process with the available resource to provide an uninterrupted service the author suggest the method of check point and restart.…”

Section: Introductionmentioning

confidence: 99%

An Innovative Approach for Cloud-Based Web Dev App Migration

Sunil

Raghunandan

Ranjit

et al. 2021

ICT With Intelligent Applications

View full text Add to dashboard Cite

In computing, process management is a key challenge where the process is shifted from one node to another. In the distributed computing architecture, this is an important activity as all the running processes are managed and distributed among available resources. Hence, resource utilization will be done efficiently. Like in multicore machines, process migration happens based on schedule, and it is normally done since the migration happens within a system. In this paper, we aim to implement the process migration in a distributed system using CRIU and DMTCP, and also, we compare the efficiency of process migration based on time. Process migration is a mechanism of transferring a process's state from one system to another for the process to execute on the target system. Checkpoint and restart are multiplatform techniques for increasing performance. It is mainly used in Linux-based operating system. Using the concept of checkpoint-restart in userspace (CRIU), many running processes will be migrated. To indicate the state of the art in checkpoint-restart, we surveyed existing implementations. This paper presents the results of this survey and understanding the migration concept for our further works.

show abstract

“…(See Section 2.4 for a discussion of MPI checkpoint-restart services.) Kangarlou et al [51] and Garg et al [39] each show how to take a distributed snapshot of VMs. Kangarlou et al base this on a modification of Xen's live migration, while Garg et al employ DMTCP to take a distributed snapshot of KVM/QEMU VMs.…”

Section: Fault Tolerance and Efficiency In The Cloudmentioning

confidence: 99%

“…Kangarlou et al base this on a modification of Xen's live migration, while Garg et al employ DMTCP to take a distributed snapshot of KVM/QEMU VMs. The last three investigations ( [61], [51], [39]) contrasts with the cloud-agnostic (and applicationagnostic) approach employed here by directly checkpointing the processes along with their network connections.…”

Section: Fault Tolerance and Efficiency In The Cloudmentioning

confidence: 99%

See 1 more Smart Citation

Transparent checkpointing over RDMA-based networks

Cao¹

View full text Add to dashboard Cite

Fault tolerance for large-scale applications has long been an area of active research, as the size of the computation keeps growing. One of the components of a faulttolerance strategy is checkpointing. However, no explicit checkpoint-restart solution has been available for applications running over RDMA-based networks. RDMA-based networks are the primary network used in high-performance computing, and many researchers believe that RDMA networks will be widely deployed in the Cloud as the costs decrease. Existing approaches often rely on a solution that is specific to the particular MPI implementation or other parallel model in order to disconnect the network at checkpoint time, and to reconnect the network at restart time. Such schemes are difficult to incorporate for new parallel programming models, and also imply higher checkpoint overhead.In this dissertation, we present the first transparent, system-initiated checkpointrestart solution that directly supports RDMA networks. This new approach does not depend on a specific parallel programming model, and does not require any modification to the operating system. In addition, network connections remain active during checkpointing, thus making checkpointing more efficient.Conceptually, this dissertation can be divided into three parts. First, we introduce a new, generic model for RDMA networks, by extracting the key components for checkpointing an RDMA network. These components are the essential states that need to be saved, in order to restore the network connection on restart. This model is then applied to two distinct RDMA networks: InfiniBand, and Intel OmniPath. This work demonstrates the generality of the model, and it also describes variations needed to adapt to InfiniBand or Omni-Path.Second, we demonstrate the performance of the proposed approach. Moving from a medium-sized academic computer cluster to a petascale supercomputer, we show what issues are exposed as the application scales up, and how these issues are addressed. In particular, different strategies to drain the network at checkpoint time are investigated, based on the underlying network protocol. As a result, failure-free overhead is reduced to below 1%, even at the largest scale demonstrated: 32,752 processes.Third, we show how to retrofit transparent checkpointing into the Cloud, as RDMA networks are also becoming more popular in the Cloud. A Checkpointing as a Service approach is presented, which employs checkpointing to provide fault tolerance as a service in the Cloud, and enables application migration in heterogeneous cloud environments.

show abstract

Checkpoint-restart for a network of virtual machines

Cited by 13 publications

References 14 publications

Checkpointing as a Service in Heterogeneous Cloud Environments

Checkpointing as a Service in Heterogeneous Cloud Environments

An Innovative Approach for Cloud-Based Web Dev App Migration

Transparent checkpointing over RDMA-based networks

Contact Info

Product

Resources

About