A Performance Comparison of Container-Based Virtualization Systems for MapReduce Clusters

Xavier, Miguel G.; Neves, Marcelo Veiga; Rose, César A. F. De

doi:10.1109/pdp.2014.78

Cited by 76 publications

(44 citation statements)

References 13 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This finding coupled with the findings of Xavier et al [9,10] establish the premise that, for HPC scientific environments, the absence of performance interference cannot be guaranteed.…”

Section: Related Workmentioning

confidence: 52%

“…In this paper, the Montage GALFA application shrinks five (5) data cubes by averaging different number of planes (5,10,15,20,25) and then aggregates them into a mosaic [30] following three major steps.…”

Section: Interference-aware Scheduling For Dynamic Workloadsmentioning

confidence: 99%

“…Efficient resource allocation to the containers in a manner that guarantees high performance is thus of utmost importance. Correspondingly, performance evaluations of containers by Xavier et al [9,10] establish the need for an efficient isolation layer to facilitate resource sharing in HPC environments. From their experiments Xen had better isolation than containers due to nonshared Operating System (OS) indicating potential performance interference when corunning applications in container-based virtualization systems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mitigating Interference between Scientific Applications in OS-Level Virtualized Environments

Adufu

Kim

2018

Scientific Programming

View full text Add to dashboard Cite

Recent research and production environments are deploying more container technologies for the execution of HPC applications and for reproducing scientific workflows or computing environments. Research works, however, have not accounted for performance interference when executing corunning applications in containers though the absence of an efficient performance isolation layer cannot guarantee the absence of performance interference among multiple corunning applications which share resources. In this research, we propose an interference-aware scheduling method that mitigates the problem of performance interference based on applications' I/O and CPU usage profiles. The proposed method estimates the amount of interference between various pairs of applications and coschedules them based on estimated interference. We evaluate the proposed method for both Bag-of-Tasks (BOT) scientific applications and scientific workflows and compare our method to the Weighted Mean Method. Our method improves the performance of the target scientific application by coscheduling applications with the least estimated interference ratios.

show abstract

“…This finding coupled with the findings of Xavier et al [9,10] establish the premise that, for HPC scientific environments, the absence of performance interference cannot be guaranteed.…”

Section: Related Workmentioning

confidence: 52%

Section: Interference-aware Scheduling For Dynamic Workloadsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mitigating Interference between Scientific Applications in OS-Level Virtualized Environments

Adufu

Kim

2018

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…we created a blkio cgroup [85] which controls the I/O policies enforced by the LXC process. The test included write operation of 5 GiB.…”

Section: Storage Footprint and Usagementioning

confidence: 99%

Cloud Computing Cost and Energy Optimization through Federated Cloud SoS

2017

View full text Add to dashboard Cite

CLOUD COMPUTING COST AND ENERGY OPTIMIZATION THROUGH FEDERATED CLOUD SOSThe two most significant differentiators amongst contemporary Cloud Computing service providers have increased green energy use and datacenter resource utilization. This work addresses these two issues from a system's architectural optimization viewpoint. The proposed approach herein, allows multiple cloud providers to utilize their individual computing resources in three ways by:(1) cutting the number of datacenters needed, (2) scheduling available datacenter grid energy via aggregators to reduce costs and power outages, and lastly by (3) utilizing, where appropriate, more renewable and carbon-free energy sources. Altogether our proposed approach creates an alternative paradigm for a Federated Cloud SoS approach. The proposed paradigm employs a novel control methodology that is tuned to obtain both financial and environmental advantages. It also supports dynamic expansion and contraction of computing capabilities for handling sudden variations in service demand as well as for maximizing usage of time varying green energy supplies. Herein we analyze the core SoS requirements, concept synthesis, and functional architecture with an eye on avoiding inadvertent cascading conditions. We suggest a physical architecture that diminishes unwanted outcomes while encouraging desirable results. Finally, in our approach, the constituent cloud services retain their independent ownership, objectives, funding, and sustainability means.This work analyzes the core SoS requirements, concept synthesis, and functional architecture.It suggests a physical structure that simulates the primary SoS emergent behavior to diminish unwanted outcomes while encouraging desirable results. The report will analyze optimal computing generation methods, optimal energy utilization for computing generation as well as a procedure for building optimal datacenters using a unique hardware computing system design based on the openCompute community as an illustrative collaboration platform. Finally, the research concludes with ii security features cloud federation requires to support to protect its constituents, its constituents tenants and itself from security risks.iii ACKNOWLEDGEMENTS

show abstract

“…Concerning the creation of VCs based on containers, studies like [191] analyzed and compared some of the container technologies available to the community (Linux-VServer, OpenVZ and LXC) from the point of view of MapReduce workloads, executing several benchmarks to test their performance and manageability. The results show that container-based systems reached near-native performance though LXC offers the best relationship of performance and isolation.…”

Section: Virtual Computer Clustersmentioning

confidence: 99%

High Performance Scientific Computing over Hybrid Cloud Platforms

Arroyo¹

View full text Add to dashboard Cite

"Croyez ceux qui cherchent la vérité, doutez de ceux qui la trouvent." "Cree a aquellos que buscan la verdad. Duda de los que la han encontrado" André Gide (1952).iii AbstractScientific applications generally require large computational requirements, memory and data management for their execution. Such applications have traditionally used high-performance resources, such as shared memory supercomputers, clusters of PCs with distributed memory, or resources from Grid infrastructures on which the application needs to be adapted to run successfully. In recent years, the advent of virtualization techniques, together with the emergence of Cloud Computing, has caused a major shift in the way these applications are executed. However, the execution management of scientific applications on high performance elastic platforms is not a trivial task.

show abstract

A Performance Comparison of Container-Based Virtualization Systems for MapReduce Clusters

Cited by 76 publications

References 13 publications

Mitigating Interference between Scientific Applications in OS-Level Virtualized Environments

Mitigating Interference between Scientific Applications in OS-Level Virtualized Environments

Cloud Computing Cost and Energy Optimization through Federated Cloud SoS

High Performance Scientific Computing over Hybrid Cloud Platforms

Contact Info

Product

Resources

About