Improving the Network Performance of a Container-Based Cloud Environment for Hadoop Systems

Rista, Cassiano; Griebler, Dalvan; Maron, Carlos A. F.; Fernandes, Luiz Gustavo

doi:10.1109/hpcs.2017.97

Cited by 14 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cloud computing is a robust environment to perform largescale and complex computations as it provides security, efficiency, flexibility, pay-as-you-go billing structure, and a scalable data storage [16], [17]. As a result, many organizations have explored it to support data-intensive applications and services in the most diverse domains.…”

Section: State-of-the-art In Adaptive Stream Processingmentioning

confidence: 99%

Boosting Big Data Streaming Applications in Clouds With BurstFlow

et al. 2020

View full text Add to dashboard Cite

Section: State-of-the-art In Adaptive Stream Processingmentioning

confidence: 99%

Boosting Big Data Streaming Applications in Clouds With BurstFlow

et al. 2020

View full text Add to dashboard Cite

“…Rista et al [9] create an evaluation methodology of performance measurements like bandwidth, throughput, latency and completion times of Hadoop applications. In its evaluation, they employed the Network Bonding mode 4 (IEEE 802.3ad) and executed Hadoop up to 3 instances concurrently in LXC containers.…”

Section: Related Workmentioning

confidence: 99%

“…To the best of our knowledge, there is no other paper in the literature that performs the same studies. The closest investigation is from Rista et al [9], which performs NIC aggregation with bonding mode 4 (802.3ad) and its performance was verified to a specific application from the big data field. Our paper extends the contributions of this previous work.…”

Section: Introductionmentioning

confidence: 99%

Minimizing Communication Overheads in Container-based Clouds for HPC Applications

Maliszewski

Vogel

Griebler³

et al. 2019

2019 IEEE Symposium on Computers and Communications (ISCC)

Self Cite

View full text Add to dashboard Cite

Although the industry has embraced the cloud computing model, there are still significant challenges to be addressed concerning the quality of cloud services. Network-intensive applications may not scale in the cloud due to the sharing of the network infrastructure. In the literature, performance evaluation studies are showing that the network tends to limit the scalability and performance of HPC applications. Therefore, we proposed the aggregation of Network Interface Cards (NICs) in a ready-touse integration with the OpenNebula cloud manager using Linux containers. We perform a set of experiments using a network microbenchmark to get specific network performance metrics and NAS parallel benchmarks to analyze the performance impact on HPC applications. Our results highlight that the implementation of NIC aggregation improves network performance in terms of throughput and latency. Moreover, HPC applications have different patterns of behavior when using our approach, which depends on communication and the amount of data transferring. While network-intensive applications increased the performance up to 38%, other applications with aggregated NICs maintained the same performance or presented slightly worse performance.

show abstract

“…The Cloud Service Provider (CSP) manages its own physical resources. This interface might vary depending on the provider, but maintains properties like elasticity, isolation and flexibility [3].…”

Section: Introductionmentioning

confidence: 99%

Enabling Strategies for Big Data Analytics in Hybrid Infrastructures

Anjos

Matteussi

Souza

et al. 2018

2018 International Conference on High Performance Computing &Amp; Simulation (HPCS)

View full text Add to dashboard Cite

A huge volume of data is produced every day by social networks (e.g. Facebook, Instagram, Whatsapp, etc.), sensors, mobile devices and other applications. Although the Cloud computing scenario has grown rapidly in recent years, it still suffers from a lack of the kind of standardization that involves the resource management for Big Data applications, such as the case of MapReduce. In this context, the users face a big challenge in attempting to understand the requirements of the application and how to consolidate the resources properly. This scenario raises significant challenges in the different areas: systems, infrastructure, platforms as well as providing several research opportunities in Big Data Analytics. This work proposes the use of hybrid infrastructures such as Cloud and Volunteer Computing for Big Data processing and analysis. In addition, it provides a data distribution model that improves the resource management of Big Data applications in hybrid infrastructures. The results indicate the feasibility of hybrid infrastructures since it supports the reproducibility and predictability of Big Data processing by low and high-scale simulation within Hybrid infrastructures.

show abstract

Improving the Network Performance of a Container-Based Cloud Environment for Hadoop Systems

Cited by 14 publications

References 15 publications

Boosting Big Data Streaming Applications in Clouds With BurstFlow

Boosting Big Data Streaming Applications in Clouds With BurstFlow

Minimizing Communication Overheads in Container-based Clouds for HPC Applications

Enabling Strategies for Big Data Analytics in Hybrid Infrastructures

Contact Info

Product

Resources

About