Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed

Bolze, Raphaël; Cappello, Franck; Caron, Eddy; Daydé, Michel; Desprez, Frédéric; Jeannot, Emmanuel; Jégou, Yvon; Lanteri, Stéphane; Leduc, Julien; Melab, N.; Mornet, Guillaume; Namyst, Raymond; Primet, Pascale Vicat-Blanc; Quétier, Benjamin; Richard, Olivier; Talbi, El‐Ghazali; Touche, Iréa

doi:10.1177/1094342006070078

Cited by 379 publications

(257 citation statements)

References 11 publications

Supporting

Mentioning

248

Contrasting

Unclassified

Order By: Relevance

“…The experimental platform used to run our experiments is Grid'5000 [8]. For the purpose of this work, We reserved 100 nodes of the graphene cluster.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Leveraging Collaborative Content Exchange for On-Demand VM Multi-deployments in IaaS Clouds

Nicolae

Rafique

2013

Euro-Par 2013 Parallel Processing

View full text Add to dashboard Cite

Abstract. A critical feature of IaaS cloud computing is the ability to deploy, boot and terminate large groups of inter-dependent VMs very quickly, which enables users to efficiently exploit the on-demand nature and elasticity of clouds even for large-scale deployments. A common pattern in this context is multi-deployment, i.e., using the same VM image template to instantiate a large number of VMs in parallel. A difficult trade-off arises in this context: access the content of the template on-demand but slowly due to I/O bottlenecks or pre-broadcast the full contents of the template on the local storage of the hosting nodes to avoid such bottlenecks. Unlike previous approaches that are biased towards either of the extremes, we propose a scheme that augments on-demand access through a collaborative scheme in which the VMs aim to leverage the similarity of access pattern in order to anticipate future accesses and exchange chunks between themselves in an attempt to reduce contention to the remote storage where the VM image template is stored. Large scale experiments show improvements in read throughput between 30%-40% compared to on-demand access schemes that perform in isolation.

show abstract

“…The experimental platform used to run our experiments is Grid'5000 [8]. For the purpose of this work, We reserved 100 nodes of the graphene cluster.…”

Section: Methodsmentioning

confidence: 99%

“…This is functionally equivalent to the broadcast technique but completely removes the broadcast overhead (Section 3.4). -We experimentally evaluate the benefits of our approach on the Grid5000 [8] testbed by performing multi-deployments on dozens of nodes (Section 4).…”

Section: Introductionmentioning

confidence: 99%

Leveraging Collaborative Content Exchange for On-Demand VM Multi-deployments in IaaS Clouds

Nicolae

Rafique

2013

Euro-Par 2013 Parallel Processing

View full text Add to dashboard Cite

show abstract

“…We carried out experiments on a 10-nodes cluster called Borderline, part of the Grid5000 testbed [9]. Borderline nodes are similar to the exemple depicted by figure 1: each node features four dies (a 2.6 GHz AMD Opteron 2218) with two cores each.…”

Section: Experimental Environment and Nas Benchmarks Choicementioning

confidence: 99%

Towards an Efficient Process Placement Policy for MPI Applications in Multicore Environments

Mercier

Clet-Ortega

2009

Recent Advances in Parallel Virtual Machine and Message Passing Interface

View full text Add to dashboard Cite

Abstract. This paper presents a method to efficiently place MPI processes on multicore machines. Since MPI implementations often feature efficient supports for both shared-memory and network communication, an adequate placement policy is a crucial step to improve applications performance. As a case study, we show the results obtained for several NAS computing kernels and explain how the policy influences overall performance. In particular, we found out that a policy merely increasing the intranode communication ratio is not enough and that cache utilization is also an influential factor. A more sophisticated policy (eg. one taking into account the architecture's memory structure) is required to observe performance improvements.

show abstract

“…It is also supported by the ANR OverFlow project (ANR-15-CE25-0003). The experiments presented in this paper were carried out on the Grid'5000 testbed [49].…”

Section: Acknowledgmentmentioning

confidence: 99%

Spark Versus Flink: Understanding Performance in Big Data Analytics Frameworks

Marcu

Costan

Antoniu

et al. 2016

2016 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Abstract-Big Data analytics has recently gained increasing popularity as a tool to process large amounts of data on-demand. Spark and Flink are two Apache-hosted data analytics frameworks that facilitate the development of multi-step data pipelines using directly acyclic graph patterns. Making the most out of these frameworks is challenging because efficient executions strongly rely on complex parameter configurations and on an in-depth understanding of the underlying architectural choices. Although extensive research has been devoted to improving and evaluating the performance of such analytics frameworks, most of them benchmark the platforms against Hadoop, as a baseline, a rather unfair comparison considering the fundamentally different design principles. This paper aims to bring some justice in this respect, by directly evaluating the performance of Spark and Flink. Our goal is to identify and explain the impact of the different architectural choices and the parameter configurations on the perceived end-to-end performance. To this end, we develop a methodology for correlating the parameter settings and the operators execution plan with the resource usage. We use this methodology to dissect the performance of Spark and Flink with several representative batch and iterative workloads on up to 100 nodes. Our key finding is that there none of the two framework outperforms the other for all data types, sizes and job patterns. This paper performs a fine characterization of the cases when each framework is superior, and we highlight how this performance correlates to operators, to resource usage and to the specifics of the internal framework design.

show abstract

Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed

Cited by 379 publications

References 11 publications

Leveraging Collaborative Content Exchange for On-Demand VM Multi-deployments in IaaS Clouds

Leveraging Collaborative Content Exchange for On-Demand VM Multi-deployments in IaaS Clouds

Towards an Efficient Process Placement Policy for MPI Applications in Multicore Environments

Spark Versus Flink: Understanding Performance in Big Data Analytics Frameworks

Contact Info

Product

Resources

About