Scaling modeling and simulation on high-performance computing clusters

Mikailov, Mike; Qiu, Junshan; Luo, Fu-Jyh; Whitney, Stephen; Petrick, Nicholas

doi:10.1177/0037549719878249

Cited by 4 publications

(2 citation statements)

References 31 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The techniques presented in this work are generally applicable to scaling large-scale image processing problems and a wide variety of applications in bioinformatics, modeling, and simulation. For example, we have applied our parallelization technique to next generation sequencing data for alignment and search of biological sequences [50], drug-protein interaction data for investigating how 3,100 active drug ingredients can be expected to interact at a molecular level with 10,000 proteins known to exist in the human body [51], and Markov Chain Monte Carlo simulations [52]. Fig.…”

Section: General Applicabilitymentioning

confidence: 99%

Scaling the Inference of Digital Pathology Deep Learning Models Using CPU-Based High-Performance Computing

Mikailov

Chen

2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

Digital pathology whole-slide images (WSIs) are large-size gigapixel images and image analysis based on deep learning artificial intelligence (AI) technology often involves pixelwise testing of a trained deep learning neural network (DLNN) on hundreds of WSI images, which is time consuming. We take advantage of High-Performance Computing (HPC) facilities to parallelize this procedure into multiple independent (and hence delightfully parallel) tasks. However, traditional software parallelization techniques and regular file formats can have significant scaling problems on HPC clusters. In this work, a useful computational strategy is designed to localize and extract relevant patches in WSI files and group them in HDF5 files well suited for parallel I/O. HPC's array job facilities are adapted for hierarchical scaling and parallelization of WSI pre-processing and testing of trained algorithms. Applying these techniques to testing a trained DLNN on the CAMELYON datasets with 399 WSIs reduced the theoretical processing time of 18 years on a single CPU or 30 days on a single GPU to less than 45 hours on an HPC cluster of 4,000 CPU cores. The efficiency-accuracy trade-off we demonstrated on this dataset further reinforced the importance of efficient computation techniques, without which the accuracy may be sacrificed. The framework developed here for testing DLNNs does not rely on any specific neural network architecture and HPC cluster setup and can be utilized for any large-scale image processing and big-data analysis. Impact Statement -The exponential growth of imaging and other biomedical data along with advanced AI technologies hold great promise to revolutionize medicine. However, handling big data and intensive computations represent a tremendous challenge. Our techniques of data handling and computations parallelization provide a seamless and efficient solution for imaging and other big-data applications. Specifically, we demonstrated the usefulness of our parallelization technique coupled with the HDF5 file handling for processing large amount of image data in a high-performance computing environment. Such a technique is particularly useful when an HPC facility is readily available and GPU resources are limited. Moreover, when a computation task involves code that has been programmed only for CPU, the CPU code can limit the use rate of GPU thereby becoming the bottleneck of the computational job. Although one could run the two parts in an HPC facility and a GPU facility separately, this would need extra storage space for saving data and extra overhead time for file I/O and data transfer. Our technique as a seamless and efficient solution is generally useful for big-data and computationally intensive applications.

show abstract

Section: General Applicabilitymentioning

confidence: 99%

Scaling the Inference of Digital Pathology Deep Learning Models Using CPU-Based High-Performance Computing

Mikailov

Chen

2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…With these measures, we extend the typical performance evaluation of clusters toward the consideration of overhead analysis in its different forms, total parallel overhead, cluster overhead, and coupling. By quantifying these overheads, HPC applications can be parameterized to be executed in an alternative programming model, for instance, hybrid parallelization MPI + OpenMP; 7 also, alternative algorithmic approaches can be adopted to face the scalability issues that suggest these overheads; 16 finally, non-HPC clusters can be adjusted to decrease these overheads.…”

Section: Related Workmentioning

confidence: 99%

Estimating the overhead and coupling of scientific computing clusters

Vivas

Castro

2021

SIMULATION

View full text Add to dashboard Cite

Since simulation became the third pillar of scientific research, several forms of computers have become available to drive computer aided simulations, and nowadays, clusters are the most popular type of computers supporting these tasks. For instance, cluster settings, such as the so-called supercomputers, cluster of workstations (COW), cluster of desktops (COD), and cluster of virtual machines (COV) have been considered in literature to embrace a variety of scientific applications. However, those scientific applications categorized as high-performance computing (HPC) are conceptually restricted to be addressed only by supercomputers. In this aspect, we introduce the notions of cluster overhead and cluster coupling to assess the capacity of non-HPC systems to handle HPC applications. We also compare the cluster overhead with an existing measure of overhead in computing systems, the total parallel overhead, to explain the correctness of our methodology. The evaluation of capacity considers the seven dwarfs of scientific computing, which are well-known, scientific computing building blocks considered in the development of HPC applications. The evaluation of these building blocks provides insights regarding the strengths and weaknesses of non-HPC systems to deal with future HPC applications developed with one or a combination of these algorithmic building blocks.

show abstract