GPU-Accelerated Machine Learning Inference as a Service for Computing in Neutrino Experiments

Abstract

Machine learning algorithms are becoming increasingly prevalent and performant in the reconstruction of events in accelerator-based neutrino experiments. These sophisticated algorithms can be computationally expensive. At the same time, the data volumes of such experiments are rapidly increasing. The demand to process billions of neutrino events with many machine learning algorithm inferences creates a computing challenge. We explore a computing model in which heterogeneous computing with GPU coprocessors is made available as a web service. The coprocessors can be efficiently and elastically deployed to provide the right amount of computing for a given processing task. With our approach, Services for Optimized Network Inference on Coprocessors (SONIC), we integrate GPU acceleration specifically for the ProtoDUNE-SP reconstruction chain without disrupting the native computing workflow. With our integrated framework, we accelerate the most time-consuming task, track and particle shower hit identification, by a factor of 17. This results in a factor of 2.7 reduction in the total processing time when compared with CPU-only production. For this particular task, only 1 GPU is required for every 68 CPU threads, providing a cost-effective solution.

Machine Learning in the Search for New Fundamental Physics

“…With ML becoming increasingly common in neutrino experiments, the community is further steering its attention toward hardware acceleration of ML-based inference. GPU-accelerated ML inference as a service for computing in neutrino experiments is discussed in [229], while new developments are also targeting GPUor FPGA-based acceleration for use of machine learning algorithms such as 1D or 2D CNNs in real-time or online processing of raw LArTPC data at the data acquisition and trigger level [197][198][199].…”

Charged Particle Tracking via Edge-Classifying Interaction Networks

“…Other heterogeneous computing resources specialized for inference may be even more beneficial. This speed-up may benefit the experiments' computing workflows by accessing these resources as an on-demand, scalable service [46][47][48].…”

Hardware-accelerated Inference for Real-Time Gravitational-Wave Astronomy

“…Significant motivation is taken from the integration of these concepts for usage in high energy physics (HEP) where recent algorithmic advances and the availability of large datasets have greatly facilitated the adoption of ML. Previous work with experiments at the CERN Large Hadron Collider and the ProtoDUNE-SP experiment at the Fermi National Accelerator Laboratory have shown that the as-a-service computing model has the potential to offer impressive speed-ups, improved performance, and reduced complexity relative to traditional computing models (Krupa 2021;Rankin et al 2020;Wang et al 2020). These works have also demonstrated the ability to perform inference as-a-service with both GPUs as well as FPGAs.…”

“…In order to take full advantage of accelerators, modifications must be made to the standard model of computing, in which pipelines directly manage the accelerated resources they use for execution. An alternative model, which has gained popularity in other fields, is called "as-a-service" (Krupa 2021;Wang et al 2020). When used to specifically denote accelerated ML inference, it is referred to as Inference-as-a-Service (IaaS).…”