Scalable computing in Java with PCJ Library. Improved collective operations

Abstract

Machine learning and Big Data workloads are becoming as important as traditional HPC ones. AI and Big Data users tend to use new programming languages such as Python, Julia, or Java, while the HPC community is still dominated by C/C++ or Fortran. Hence, there is a need for new programming libraries and languages that will integrate different applications and allow them to run on large computer infrastructure. Since modest computers are multinode and multicore, parallel execution is an additional challenge here. For that purpose, we have developed the PCJ library, which introduces parallel programming capabilities to Java using the Partitioned Global Address Space model. It does not modify language nor running environment (JVM). The PCJ library allows for easy development of parallel code and runs it on laptops, workstations, supercomputers, and the cloud. This paper presents an overview of the PCJ library and its usage in parallelizing selected workloads, including HPC, AI, and Big Data. The performance and scalability are presented. We present recent addition to the PCJ library, which are collective operations. The collective operations significantly reduce the number of lines of code to write, ensuring good performance.

Performance evaluation of Java/PCJ implementation of parallel algorithms on the cloud (extended version)

“…Results presented hereinafter use a simple serial reduction scheme. Performance can be enhanced with the use of a hypercube‐based reduction, as presented in our other works 35,37 . Full source code and sample input data are available on GitHub 53…”

“…The library offers a method to create subgroups of the PCJ threads (splitGroup(…)). Creating subgroups is especially useful when using collective operations like reduce(…), collect(…) or scatter(…) which are described in Reference 35. Each communication method has also an asynchronous counterpart with the async prefix in the name (e.g., asyncGet(…)).…”

“…A detailed description of the library and its methods is available in our previous publications 2,3,35 …”

“…Full source code is available on GitHub. 49 Performance results of these implementations are presented in Figure 2. All three implementations were executed on the Cray XC40 system to select the best performing one for further tests.…”

“…Performance can be enhanced with the use of a hypercube-based reduction, as presented in our other works. 35,37 Full source code and sample input data are available on GitHub. 53 For the performance tests, we have used two text files of novels in a weak scalability regime-each thread processes the whole file, so increasing the number of threads gives more data to process.…”