Multi-step image compositing for massively parallel rendering

Abstract

High performance visualization has played an important role in computer-aided scientific discovery and has become an indispensable tool for computational scientists. Sortlast parallel rendering is a proven approach for visual data analytics by extracting meaningful information from huge data sets generated from large scale scientific computing. Image compositing is the last stage of sort-last parallel rendering pipeline and works by combining the images generated by the rendering nodes to generate the final image. Since it requires interprocess communication among the entire nodes, it usually dominates the total cost of the parallel rendering process. In current high-end massively parallel HPC systems, where tens or even hundreds of thousands of nodes can be involved, performance degradation is inevitable even using theoretically scalable image compositing algortithms such as the well-known Binary-Swap method. To minimize this undesirable performance degradation, we propose the multi-step image compositing method, where the image compositing nodes are divided into smaller groups and the entire process is performed in several steps. We evaluated the proposed image compositing method on RIKEN K computer, which is a massively parallel HPC system, and we obtained encouraging results showing the effectiveness of this method in a large-scale image compositing environment. We also foresee a great potential of this method to meet the large-scale image compositing demands brought about by the rapid increase in processor counts of current and next-generation HPC systems.

Multi-Step Image Composition Approach for Sort-Last Massively Parallel Rendering

“…In massively parallel rendering environments where tens of thousands of rendering nodes are involved, it is important to have a scalable image composition code for this degree of parallelism. However, scalability problems on massively parallel image composition environments have already been reported on different HPC systems such as T2K Open Supercomputer [9], IBM Blue Gene/L [10], IBM Blue Gene/P [11], and K Computer [12]. On K computer, besides this performance degradation when using a large composition node counts, there is a more critical issue since MPI Gatherv, an MPI collective function used to implement these aforementioned image composition algorithms, does not work over 50K composition node counts due to the buffer error (MRQ Overflow) in the current MPI implementation for K computer.…”

“…Although it might be a temporary problem, large-scale parallel image composition using 64K composition nodes, such as presented in [11], is not directly possible. In this paper, we focused on the Multi-Step approach presented in [12] and expanded the investigation including floating point pixel format for attending high-quality rendering requirements; a method for selecting the maximum group size to be used in each…”

“…The idea of Multi-Step image composition first appeared in [10] where different image composition approaches could be combined (Binary-Swap with Direct Sent and Binary Tree) on IBM Blue Gene/L supercomputer. In a more recent evaluation, Binary-Swap only Multi-Step approach has been evaluated on K computer using 32-bit RGBA images [12]. In this paper, we made a more profound investigation extendeding its use for 128-bit RGBA images, with floating point color and transparency information, required in high-quality volume rendering.…”

“…We evaluated the Multi-Step image composition approach utilizing the group sizes defined by the threshold of 10% sizes different from the aforementioned sizes, we can use the image size in between as the threshold for group size selection as shown in [12]. For instance, the group size of 1024 will be valid for any image sizes between 1.5 and 6 MPixels.…”

Analyzing the I/O Performance of Post-Hoc Visualization of Huge Simulation Datasets on the K Computer

“…As the scale of computational science simulations grows to deal with increasingly complex problems, we can also verify a significant increase in the volume of data produced [Roten et al 2016]. To derive meaningful information from these huge datasets, leading to scientific discoveries and breakthroughs, scientists and engineers rely upon large-scale visualization and data analysis systems [Nonaka et al 2014]. Such dataintensive applications pose a great pressure on the shared backend storage system of modern high performance computing (HPC) environments.…”