Scaling the Inference of Digital Pathology Deep Learning Models Using CPU-Based High-Performance Computing

Abstract

Digital pathology whole-slide images (WSIs) are large-size gigapixel images and image analysis based on deep learning artificial intelligence (AI) technology often involves pixelwise testing of a trained deep learning neural network (DLNN) on hundreds of WSI images, which is time consuming. We take advantage of High-Performance Computing (HPC) facilities to parallelize this procedure into multiple independent (and hence delightfully parallel) tasks. However, traditional software parallelization techniques and regular file formats can have significant scaling problems on HPC clusters. In this work, a useful computational strategy is designed to localize and extract relevant patches in WSI files and group them in HDF5 files well suited for parallel I/O. HPC's array job facilities are adapted for hierarchical scaling and parallelization of WSI pre-processing and testing of trained algorithms. Applying these techniques to testing a trained DLNN on the CAMELYON datasets with 399 WSIs reduced the theoretical processing time of 18 years on a single CPU or 30 days on a single GPU to less than 45 hours on an HPC cluster of 4,000 CPU cores. The efficiency-accuracy trade-off we demonstrated on this dataset further reinforced the importance of efficient computation techniques, without which the accuracy may be sacrificed. The framework developed here for testing DLNNs does not rely on any specific neural network architecture and HPC cluster setup and can be utilized for any large-scale image processing and big-data analysis. Impact Statement -The exponential growth of imaging and other biomedical data along with advanced AI technologies hold great promise to revolutionize medicine. However, handling big data and intensive computations represent a tremendous challenge. Our techniques of data handling and computations parallelization provide a seamless and efficient solution for imaging and other big-data applications. Specifically, we demonstrated the usefulness of our parallelization technique coupled with the HDF5 file handling for processing large amount of image data in a high-performance computing environment. Such a technique is particularly useful when an HPC facility is readily available and GPU resources are limited. Moreover, when a computation task involves code that has been programmed only for CPU, the CPU code can limit the use rate of GPU thereby becoming the bottleneck of the computational job. Although one could run the two parts in an HPC facility and a GPU facility separately, this would need extra storage space for saving data and extra overhead time for file I/O and data transfer. Our technique as a seamless and efficient solution is generally useful for big-data and computationally intensive applications.