GPU simulation with Opticks: The future of optical simulations for LZ

Abstract

The LZ collaboration aims to directly detect dark matter by using a liquid xenon Time Projection Chamber (TPC). In order to probe the dark matter signal, observed signals are compared with simulations that model the detector response. The most computationally expensive aspect of these simulations is the propagation of photons in the detector’s sensitive volume. For this reason, we propose to offload photon propagation modelling to the Graphics Processing Unit (GPU), by integrating Opticks into the LZ simulations workflow. Opticks is a system which maps Geant4 geometry and photon generation steps to NVIDIA’s OptiX GPU raytracing framework. This paradigm shift could simultaneously achieve a massive speed-up and an increase in accuracy for LZ simulations. By using the technique of containerization through Shifter, we will produce a portable system to harness the NERSC supercomputing facilities, including the forthcoming Perlmutter supercomputer, and enable the GPU processing to handle different detector configurations. Prior experience with using Opticks to simulate JUNO indicates the potential for speed-up factors over 1000× for LZ, and by extension other experiments requiring photon propagation simulations.

GPU-based optical simulation of the DARWIN detector

“…The implementation of the optical simulation is commonly done using the Geant4 toolkit [14,19,20]. It has been estimated that Geant4 photon tracking consumes >95% of CPU time used in simulations of a current-generation dual-phase LXe TPC detector [21]. Recently, alternative MC frameworks are being considered for photon tracking by individual groups [22][23][24] and large experiments [17,21] in the LXe community, offering a faster simulation and/or a more convenient geometry implementation.…”

“…It has been estimated that Geant4 photon tracking consumes >95% of CPU time used in simulations of a current-generation dual-phase LXe TPC detector [21]. Recently, alternative MC frameworks are being considered for photon tracking by individual groups [22][23][24] and large experiments [17,21] in the LXe community, offering a faster simulation and/or a more convenient geometry implementation. In this work, we describe a detailed optical simulation of the DARWIN detector implemented using one such framework, Chroma [25,26].…”

GPU-based optical simulation of the DARWIN detector

“…The implementation of the optical simulation is commonly done using the Geant4 toolkit [14,19,20]. It has been estimated that Geant4 photon tracking consumes >95% of CPU time used in simulations of a current-generation dual-phase LXe TPC detector [21]. Recently, alternative MC frameworks are being considered for photon tracking by various groups [17,[21][22][23][24] in the LXe community, offering a faster simulation and/or a more convenient geometry implementation.…”

“…It has been estimated that Geant4 photon tracking consumes >95% of CPU time used in simulations of a current-generation dual-phase LXe TPC detector [21]. Recently, alternative MC frameworks are being considered for photon tracking by various groups [17,[21][22][23][24] in the LXe community, offering a faster simulation and/or a more convenient geometry implementation. In this work, we describe a detailed optical simulation of the DARWIN detector implemented using one such framework, Chroma [25,26].…”

“…Chroma is claimed to be 50 [26] to 400 [28] times faster than Geant4, which runs on central processing units (CPUs). Compared to a similar GPU-based framework [21,29], Chroma has an additional advantage of reducing the time to set up a complex simulation, which is particularly helpful during active detector development, when numerous alternative geometries need to be evaluated quickly. This is thanks to the ability to import detector geometries from computer-aided design (CAD) files.…”