Sparse Matrix-Based HPC Tomography

Abstract

Tomographic imaging has benefited from advances in X-ray sources, detectors and optics to enable novel observations in science, engineering and medicine. These advances have come with a dramatic increase of input data in the form of faster frame rates, larger fields of view or higher resolution, so high performance solutions are currently widely used for analysis. Tomographic instruments can vary significantly from one to another, including the hardware employed for reconstruction: from single CPU workstations to large scale hybrid CPU/GPU supercomputers. Flexibility on the software interfaces and reconstruction engines are also highly valued to allow for easy development and prototyping. This paper presents a novel software framework for tomographic analysis that tackles all aforementioned requirements. The proposed solution capitalizes on the increased performance of sparse matrixvector multiplication and exploits multi-CPU and GPU reconstruction over MPI. The solution is implemented in Python and relies on CuPy for fast GPU operators and CUDA kernel integration, and on SciPy for CPU sparse matrix computation. As opposed to previous tomography solutions that are tailor-made for specific use cases or hardware, the proposed software is designed to provide flexible, portable and highperformance operators that can be used for continuous integration at different production environments, but also for prototyping new experimental settings or for algorithmic development. The experimental results demonstrate how our implementation can even outperform state-of-theart software packages used at advanced X-ray sources worldwide.

Improved Acquisition and Reconstruction for Wavelength-Resolved Neutron Tomography

“…Therefore depending on the available computing resources and desired rate at which we seek feedback from the system we can select the appropriate number of slices and wavelength bins to reconstruct over. We note that this compute can be dramatically accelerated with highly optimized MBIR implementations [ 35 , 36 ]; in this paper we focus on demonstrating a proof-of-concept that highlights the quality improvements that can be achieved using the proposed system.…”

“…In practice, for the type of samples and detectors used in this paper we have observed that it is possible to obtain a useful (partial) reconstruction from the data in lesser time than it takes to measure the next projection using our implementation of the algorithm. We believe that using even more optimized implementations [ 35 , 36 ] we can obtain large hyper-spectral reconstructions in near-real time for WRNT systems using fairly small compute clusters. We plan to open-source our reconstruction code as a part of the pyMBIR package [ 37 ] upon publication of the current work.…”

mpi4py: Status Update After 12 Years of Development

“…To facilitate API interoperability and zerocopy GPU data exchange, several Python GPU libraries jointly define and implement the CUDA Array Interface (CAI) protocol, 10 an effort that originated after the initiative of the Numba project. 11 The CAI largely follows the NumPy array interface protocol 12 and requires that all compliant objects add a new Python attribute __cuda_array_interface__, con-taining the raw GPU buffer address and additional metadata. This way, the attribute filled by a producer can be correctly interpreted by a consumer.…”

“…MPI parallelization is critical in reducing memory stress and time to solution. Several image reconstruction packages developed in the computational microscopy community depend on mpi4py for parallel computing, including nsls2ptycho [9], Adorym [10], Tike, 26 PyNX [11], and XPACK [12]. In a recent version of nsls2ptycho [9], the GPU support is changed from PyCUDA to CuPy, allowing a unified CPU/GPU codebase thanks to CuPy's high compatibility with NumPy, including the support for the __array_function__ This work is licensed under a Creative Commons Attribution 4.0 License.…”

Sparse Matrix-Based HPC Tomography

“…The experimental results demonstrate how the proposed solution can reconstruct datasets of 68 GB in less than 5 seconds, even surpassing the performance of TomoPy's fastest reconstruction engine by 2.2X. All the code of this project is also open source and available at [12].…”