F-MPJ: scalable Java message-passing communications on parallel systems

Taboada, Guillermo L.; Touriño, Juan; Doallo, Ramón

doi:10.1007/s11227-009-0270-0

Cited by 30 publications

(20 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, FastMPJ [39] was also selected as representative Message-Passing in Java (MPJ) middleware [40]. In all cases, the most efficient communication mechanism available for network transfers and shared memory scenarios was selected.…”

Section: Experimental Configuration and Evaluation Methodologymentioning

confidence: 99%

Performance analysis of HPC applications in the cloud

Expósito

Taboada

Ramos

et al. 2013

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

The scalability of High Performance Computing (HPC) applications depends heavily on the efficient support of network communications in virtualized environments. However, Infrastructure as a Service (IaaS) providers are more focused on deploying systems with higher computational power interconnected via high-speed networks rather than improving the scalability of the communication middleware. This paper analyzes the main performance bottlenecks in HPC applications scalability on Amazon EC2 Cluster Compute platform: (1) evaluating the communication performance on shared memory and a virtualized 10 Gigabit Ethernet network; (2) assessing the scalability of representative HPC codes, the NAS Parallel Benchmarks, using an important number of cores, up to 512; (3) analyzing the new cluster instances (CC2), both in terms of single instance performance, scalability and costefficiency of its use; (4) suggesting techniques for reducing the impact of the virtualization overhead in the scalability of communication-intensive HPC codes, such as the direct access of the Virtual Machine to the network and reducing the number of processes per instance; and (5) proposing the combination of message-passing with multithreading as the most scalable and cost-effective option for running HPC applications on Amazon EC2 Cluster Compute platform.

show abstract

Section: Experimental Configuration and Evaluation Methodologymentioning

confidence: 99%

Performance analysis of HPC applications in the cloud

Expósito

Taboada

Ramos

et al. 2013

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, the most relevant related literature in MPJ collective communications comprises the papers that introduce MPJ/Ibis [10], MPJava [11], and Fast MPJ (F-MPJ) [4] projects. With respect to MPJ/Ibis, its collective library only implements one algorithm per primitive.…”

Section: Related Workmentioning

confidence: 99%

“…Their performance results, presented in [11], highlighted for the first time in the MPJ community the importance of choosing the appropriate collective communication algorithm according to the characteristics of the codes being executed and the hardware configuration employed. Finally, our own MPJ implementation F-MPJ [4] includes a scalable collective library implemented on top of point-to-point calls to a low-level communication device. Moreover, F-MPJ collective library implements up to three algorithms per primitive, selected statically (at compile-time).…”

Section: Related Workmentioning

confidence: 99%

“…With respect to the scalability of current MPJ collective libraries, several performance evaluations [4,9,12] have pointed out that their results are generally poor due to the use of algorithms that do not take advantage of multi-core architectures and non-blocking communications, in order to exploit data locality and overlap communications, respectively.…”

Section: Related Workmentioning

confidence: 99%

“…An unscalable algorithm can easily waste the performance provided by an efficient communication middleware. Unfortunately, current MPJ collective libraries show poor scalability [4]. This paper presents a scalable and efficient MPJ collectives library aiming to its integration in MPJ implementations in order to provide higher performance on multi-core clusters.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Design of efficient Java message-passing collectives on multi-core clusters

et al. 2010

Self Cite

View full text Add to dashboard Cite

This paper presents a scalable and efficient Message-Passing in Java (MPJ) collective communication library for parallel computing on multi-core architectures. The continuous increase in the number of cores per processor underscores the need for scalable parallel solutions. Moreover, current system deployments are usually multi-core clusters, a hybrid shared/distributed memory architecture which increases the complexity of communication protocols. Here, Java represents an attractive choice for the development of communication middleware for these systems, as it provides built-in networking and multithreading support. As the gap between Java and compiled languages performance has been narrowing for the last years, Java is an emerging option for High Performance Computing (HPC).Our MPJ collective communication library increases Java HPC applications performance on multi-core clusters: (1) providing multi-core aware collective primitives;(2) implementing several algorithms (up to six) per collective operation, whereas publicly available MPJ libraries are usually restricted to one algorithm; (3) analyzing the efficiency of thread-based collective operations; (4) selecting at runtime the most efficient algorithm depending on the specific multi-core system architecture, and the number of cores and message length involved in the collective operation; (5) supporting the automatic performance tuning of the collectives depending on the system and communication parameters; and (6) allowing its integration in any MPJ implementation as it is based on MPJ point-to-point primitives. A performance evalDesign of efficient MPJ collectives on multi-core clusters 127 uation on an InfiniBand and Gigabit Ethernet multi-core cluster has shown that the implemented collectives significantly outperform the original ones, as well as higher speedups when analyzing the impact of their use on collective communications intensive Java HPC applications. Finally, the presented library has been successfully integrated in MPJ Express (http://mpj-express.org), and will be distributed with the next release.

show abstract

Device level communication libraries for high‐performance computing in Java

Taboada

Touriño

Doallo

et al. 2011

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SUMMARYSince its release, the Java programming language has attracted considerable attention from the highperformance computing (HPC) community because of its portability, high programming productivity, and built-in multithreading and networking support. As a consequence, several initiatives have been taken to develop a high-performance Java message-passing library to program distributed memory architectures, such as clusters. The performance of Java message-passing applications relies heavily on the communications performance. Thus, the design and implementation of low-level communication devices that support messagepassing libraries is an important research issue in Java for HPC. MPJ Express is our Java message-passing implementation for developing high-performance parallel Java applications. Its public release currently contains three communication devices: the first one is built using the Java New Input/Output (NIO) package for the TCP/IP; the second one is specifically designed for the Myrinet Express library on Myrinet; and the third one supports thread-based shared memory communications. Although these devices have been successfully deployed in many production environments, previous performance evaluations of MPJ Express suggest that the buffering layer, tightly coupled with these devices, incurs a certain degree of copying overhead, which represents one of the main performance penalties. This paper presents a more efficient Java message-passing communications device, based on Java Input/Output sockets, that avoids this buffering overhead. Moreover, this device implements several strategies, both in the communication protocol and in the HPC hardware support, which optimizes Java message-passing communications. In order to evaluate its benefits, this paper analyzes the performance of this device comparatively with other Java and native message-passing libraries on various high-speed networks, such as Gigabit Ethernet, Scalable Coherent Interface, Myrinet, and InfiniBand, as well as on a shared memory multicore scenario. The reported communication overhead reduction encourages the upcoming incorporation of this device in MPJ Express (http://mpj-express.org). Copyright

show abstract

F-MPJ: scalable Java message-passing communications on parallel systems

Cited by 30 publications

References 17 publications

Performance analysis of HPC applications in the cloud

Performance analysis of HPC applications in the cloud

Design of efficient Java message-passing collectives on multi-core clusters

Device level communication libraries for high‐performance computing in Java

Contact Info

Product

Resources

About