Prospects and challenges of virtual machine migration in HPC

Pickartz, Simon; Clauss, Carsten; Breitbart, Jens; Lankes, Stefan; Monti, Antonello

doi:10.1002/cpe.4412

Cited by 6 publications

(2 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea is to zero memory regions only when a migration or a checkpoint is imminent and thus to avoid unnecessary zero writes. The glibc library needs to be modified to react upon migration / checkpoint requests (similar to the approach for the migration of MPI applications in [32]). In doing so, it would place zeros into freed memory regions only directly before a migration starts.…”

Section: Generalization Of the Approachmentioning

confidence: 99%

Zeroing memory deallocator to reduce checkpoint sizes in virtualized HPC environments

et al. 2018

Self Cite

View full text Add to dashboard Cite

Virtualization has become an indispensable tool in data centers and Cloud environments to flexibly assign virtual machines (VMs) to resources. Virtualization also becomes more and more attractive for High-performance Computing (HPC). This is mainly due to the strong isolation of VMs which enables: (1) the sharing of cluster nodes and optimization of the system's overall utilization; (2) load balancing by means of migrations due to the reduction of residual dependencies; and (3) the creation of system-level checkpoints increasing the fault tolerance in an application-transparent way. On the downside, the additional virtualization layer conceals information that is only available on the process level. This information has a direct influence on the checkpoint size which should be kept as small as possible. In this paper, we propose a novel technique for checkpoint size reduction in virtualized environments. We exploit the fact that the hypervisor detects zero pages which are omitted when capturing a checkpoint. Moreover, compression techniques are applied for a further reduction of the checkpoint size. We therefore fill freed memory regions with zeros supporting both the zero page detection and the compression. We evaluate our approach by taking the example of HPC applications. The results reveal a reduction of the checkpoint size by up to 9 % when compression is disabled in the hypervisor and up to 49 % with compression enabled. Furthermore, memory zeroing is able to reduce VM migration time by up to 10 % when compression is disabled and by up to 60 % when compression is enabled.

show abstract

Section: Generalization Of the Approachmentioning

confidence: 99%

Zeroing memory deallocator to reduce checkpoint sizes in virtualized HPC environments

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Pickartz et al dealt with fault tolerance on a supercomputer on which one or several nodes may be turned off during a computation for maintenance reasons. The authors provided solutions to be able to efficiently and transparently stop some nodes while the HPC application is running.…”

Section: Hpc and Co: This Special Issuementioning

confidence: 99%

HPC & Co strike back: Where are distributed paradigms heading toward?

Limet

Merlo

Spalazzi

2018

Concurrency and Computation

View full text Add to dashboard Cite

From the dawn of parallelization, the diffusion of High-Performance Computing (HPC) has grown exponentially until nowadays, thereby leading to the birth and development of a huge research community in this field. During the last decades, the important research results have led to extend the domain of HPC into three different dimensions, namely Architectures, Research Topics, and Application Domains. ArchitecturesTraditional HPC architectures have been sided with several wide-area distributed paradigms such as Peer-to-Peer (P2P), Grid, Cloud, Fog and PervasiveComputing. 1-3 Each of them aims new specific objectives along with the original goal of improving performance and reliability. In this paper, the different declinations of the traditional HPC are grouped into three macro-categories, namely SIMD, Multi- * , and Cluster.• SIMD. It stands for "single instruction, multiple data" according to the Flynn taxonomy. 4 This refers to computing systems where a single instruction stream operates on multiple data streams in order to perform operations that may be naturally parallelized (eg, array processors, GPUs, GPGPUs, etc). This category also includes many specialized processing units, as vector extensions of modern processors like Streaming SIMD Extensions (SSE) 5 or Advanced Vector Extensions (AVX). 6• Multi- * . It refers to many-core, multi-core, and multi-processor architectures. They are all architectures characterized by the presence of multiple instruction streams operating on multiple data streams. What distinguishes this type of architectures from other MIMD architectures ("multiple instructions, multiple data" always according to the Flynn taxonomy 4 ) is the sharing of the central memory.• Cluster. It refers to Cluster Computing, namely a group of cost-effective linked commercial off-the-shelf computers working together so closely that they act as a single machine. What distinguishes this type of architecture from other MIMD-like architectures is the absence of central memory sharing (each processing unit has its own private memory) but the presence of mass memory sharing. As a matter of fact, the current Top500 supercomputers 7 are mostly clusters.Regarding wide-area distributed paradigms, all of them are included in the MIMD category (according to the Flynn's taxonomy). What distinguishes this type of architecture from other MIMD architectures is the lack of sharing of both the central memory and the mass memory. This paper takes into consideration the main four distributed paradigms that arose from traditional HPC, namely Grid Computing, Ubiquitous Computing, Peer-to-Peer Computing, Cloud, and the emerging Fog Computing. In brief:• Grid Computing 8-10 denotes a distributed architecture that enables coordinated resource sharing for problem-solving within dynamic organizations consisting of individuals, institutions, and resources.• Ubiquitous Computing (also known as Pervasive Computing or UbiComp) 11,12 refers to making available to users computational resources, which provide information and servi...

show abstract