Coupling GPU and MPTCP to improve Hadoop/MapReduce performance

Wang, Chia-Hui; Yang, Chen-Kuei; Liao, Wei‐Chih; Chang, Ray-I; Wei, Tsao-Ta

doi:10.1109/igbsg.2016.7539430

Cited by 6 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study of Wang et al [11] proposed the applicability of the MultiPath TCP (MPTCP) protocol for improving the performance of a distributed Hadoop/MapReduce architecture. Their scenario exploited GPUs resources and shown the impact of network bottlenecks on the application's performance.…”

Section: Related Workmentioning

confidence: 99%

Minimizing Communication Overheads in Container-based Clouds for HPC Applications

Maliszewski

Vogel

Griebler³

et al. 2019

2019 IEEE Symposium on Computers and Communications (ISCC)

View full text Add to dashboard Cite

Although the industry has embraced the cloud computing model, there are still significant challenges to be addressed concerning the quality of cloud services. Network-intensive applications may not scale in the cloud due to the sharing of the network infrastructure. In the literature, performance evaluation studies are showing that the network tends to limit the scalability and performance of HPC applications. Therefore, we proposed the aggregation of Network Interface Cards (NICs) in a ready-touse integration with the OpenNebula cloud manager using Linux containers. We perform a set of experiments using a network microbenchmark to get specific network performance metrics and NAS parallel benchmarks to analyze the performance impact on HPC applications. Our results highlight that the implementation of NIC aggregation improves network performance in terms of throughput and latency. Moreover, HPC applications have different patterns of behavior when using our approach, which depends on communication and the amount of data transferring. While network-intensive applications increased the performance up to 38%, other applications with aggregated NICs maintained the same performance or presented slightly worse performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Minimizing Communication Overheads in Container-based Clouds for HPC Applications

Maliszewski

Vogel

Griebler³

et al. 2019

2019 IEEE Symposium on Computers and Communications (ISCC)

View full text Add to dashboard Cite

show abstract

Performance Impact of IEEE 802.3ad in Container-Based Clouds for HPC Applications

Maliszewski

Roloff

Griebler

et al. 2020

Computational Science and Its Applications – ICCSA 2020

View full text Add to dashboard Cite

Historically, large computational clusters have supported hardware requirements for executing High-Performance Computing (HPC) applications. This model has become out of date due to the high costs of maintaining and updating these infrastructures. Currently, computing resources are delivered as a service because of the cloud computing paradigm. In this way, we witnessed consistent efforts to migrate HPC applications to the cloud. However, if on the one hand cloud computing offers an attractive environment for HPC, benefiting from the pay-per-use model and on-demand resource allocation, on the other, there are still significant performance challenges to be addressed, such as the known network bottleneck. In this article, we evaluate the use of a Network Interface Cards (NIC) aggregation approach, using the IEEE 802.3ad standard to improve the performance of representative HPC applications executed in LXD container based-cloud. We assessed the aggregation impact using two and four NICs with three distinct transmission hash policies. Our results demonstrated that if the correct hash policy is selected, the NIC aggregation can significantly improve the performance of network-intensive HPC applications by up to 40%.

show abstract

Harnessing Cloud Computing to Power Up HPC Applications: The BRICS CloudHPC Project

Marques

Wang

et al. 2021

Computational Science and Its Applications – ICCSA 2021

View full text Add to dashboard Cite

Coupling GPU and MPTCP to improve Hadoop/MapReduce performance

Cited by 6 publications

References 7 publications

Minimizing Communication Overheads in Container-based Clouds for HPC Applications

Minimizing Communication Overheads in Container-based Clouds for HPC Applications

Performance Impact of IEEE 802.3ad in Container-Based Clouds for HPC Applications

Harnessing Cloud Computing to Power Up HPC Applications: The BRICS CloudHPC Project

Contact Info

Product

Resources

About