Message passing for GPGPU clusters: CudaMPI

Lawlor, Orion Sky

doi:10.1109/clustr.2009.5289129

Cited by 43 publications

(28 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DisMaRC, a distributed GPGPU based MapReduce framework is presented in [12]. In another work by Lawlor [13], the author analyses two new communication libraries, cudaMPI and glMPI, that provide an MPI-like message passing interface to communicate data stored on the graphics cards of a distributedmemory parallel computer. Then there are numerous examples of single applications that are ported on cluster of GPUs [14], [15], [16], [17].…”

Section: Related Workmentioning

confidence: 99%

High Performance Data Mining Using R on Heterogeneous Platforms

Kumar

Ozisikyilmaz

Liao

et al. 2011

2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PHD Forum

View full text Add to dashboard Cite

Abstract-The exponential increase in the generation and collection of data has led us in a new era of data analysis and information extraction. Conventional systems based on general-purpose processors are unable to keep pace with the heavy computational requirements of data mining techniques. High performance coprocessors like GPUs and FPGAs have the potential to handle large computational workloads. In this paper, we present a scalable framework aimed at providing a platform for developing and using high performance data mining applications on heterogeneous platforms. The framework incorporates a software infrastructure and a library of high performance kernels. Furthermore, it includes a variety of optimizations which increase the throughput of applications. The framework spans multiple technologies including R, GPUs, multi-core CPUs, MPI, and parallel-netCDF harnessing their capabilities for high-performance computations. This paper also introduces the concept of interleaving GPU kernels from multiple applications providing significant performance gain. Thus, in comparison to other tools available for data mining, our framework provides an easy-to-use and scalable environment both for application development and execution. The framework is available as a software package which can be easily integrated in the R programming environment.

show abstract

Section: Related Workmentioning

confidence: 99%

High Performance Data Mining Using R on Heterogeneous Platforms

Kumar

Ozisikyilmaz

Liao

et al. 2011

2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PHD Forum

View full text Add to dashboard Cite

show abstract

“…The cudaMPI library studies providing wrapper API functions by mixing CUDA and MPI data movement [21]. Similarly to MPI-ACC, Wang et al propose to add CUDA [2] support to MVAPICH2 [22] and optimize the internode communication for InfiniBand networks [28].…”

Section: Related Workmentioning

confidence: 99%

On the efficacy of GPU-integrated MPI for scientific applications

Aji

Panwar

et al. 2013

Proceedings of the 22nd International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

Scientific computing applications are quickly adapting to leverage the massive parallelism of GPUs in large-scale clusters. However, the current hybrid programming models require application developers to explicitly manage the disjointed host and GPU memories, thus reducing both efficiency and productivity. Consequently, GPU-integrated MPI solutions, such as MPI-ACC and MVAPICH2-GPU, have been developed that provide unified programming interfaces and optimized implementations for end-to-end data communication among CPUs and GPUs. To date, however, there lacks an in-depth performance characterization of the new optimization spaces or the productivity impact of such GPU-integrated communication systems for scientific applications.In this paper, we study the efficacy of GPU-integrated MPI on scientific applications from domains such as epidemiology simulation and seismology modeling, and we discuss the lessons learned. We use MPI-ACC as an example implementation and demonstrate how the programmer can seamlessly choose between either the CPU or the GPU as the logical communication end point, depending on the application's computational requirements. MPI-ACC also encourages programmers to explore novel application-specific optimizations, such as internode CPU-GPU communication with concurrent CPU-GPU computations, which can improve the overall cluster utilization. Furthermore, MPI-ACC internally implements scalable memory management techniques, thereby decoupling the low-level memory optimizations from the applications and making them scalable and portable across several architectures. Experimental results from a state-of-the-art cluster with hundreds of GPUs show that the MPI-ACC-driven new applicationspecific optimizations can improve the performance of an epidemiology simulation by up to 61.6% and the performance of a seismology modeling application by up to 44%, when compared with traditional hybrid MPI+GPU implementations. We conclude that GPU-integrated MPI significantly enhances programmer producPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

show abstract

“…cudaMPI [3] provided an MPI-like interface that handled the inherent memory copies from the GPU to the CPU before the CPU executed an MPI call. However, thanks to 0-copy memory 1 introduced in CUDA 3 [4], cudaMPI is redundant.…”

Section: Previous Workmentioning

confidence: 99%

Extending MPI to accelerators

Stuart

Balaji

Owens

2011

Proceedings of the 1st Workshop on Architectures and Systems for Big Data

View full text Add to dashboard Cite

Current trends in computing and system architecture point towards a need for accelerators such as GPUs to have inherent communication capabilities. We review previous and current software libraries that provide pseudo-communication abilities through direct message passing. We show how these libraries are beneficial to the HPC community, but are not forward-thinking enough. We give motivation as to why MPI should be extended to support these accelerators, and provide a road map of achievable milestones to complete such an extension, some of which require advances in hardware and device drivers.

show abstract

Message passing for GPGPU clusters: CudaMPI

Cited by 43 publications

References 9 publications

High Performance Data Mining Using R on Heterogeneous Platforms

High Performance Data Mining Using R on Heterogeneous Platforms

On the efficacy of GPU-integrated MPI for scientific applications

Extending MPI to accelerators

Contact Info

Product

Resources

About