Accelerating Number Theoretic Transformations for Bootstrappable Homomorphic Encryption on GPUs

Abstract

Homomorphic encryption (HE) draws huge attention as it provides a way of privacy-preserving computations on encrypted messages. Number Theoretic Transform (NTT), a specialized form of Discrete Fourier Transform (DFT) in the finite field of integers, is the key algorithm that enables fast computation on encrypted ciphertexts in HE. Prior works have accelerated NTT and its inverse transformation on a popular parallel processing platform, GPU, by leveraging DFT optimization techniques. However, these GPU-based studies lack a comprehensive analysis of the primary differences between NTT and DFT or only consider small HE parameters that have tight constraints in the number of arithmetic operations that can be performed without decryption. In this paper, we analyze the algorithmic characteristics of NTT and DFT and assess the performance of NTT when we apply the optimizations that are commonly applicable to both DFT and NTT on modern GPUs. From the analysis, we identify that NTT suffers from severe main-memory bandwidth bottleneck on large HE parameter sets. To tackle the main-memory bandwidth issue, we propose a novel NTT-specific on-the-fly root generation scheme dubbed on-the-fly twiddling (OT). Compared to the baseline radix-2 NTT implementation, after applying all the optimizations, including OT, we achieve 4.2× speedup on a modern GPU.

Accelerating Fully Homomorphic Encryption Through Architecture-Centric Analysis and Optimization

“…These show why the use of AVX-512 achieves an insufficient degree of performance improvement and also imply that we can substantially accelerates HE Mul by natively supporting these SIMD instructions. Previous work [49], [53] also showed that SIMD is effective in accelerating NTT and iNTT on CPUs and GPUs. Impact of Q on the characteristics of HE Mul: Q determines multiplicative depth L; a larger depth requires a bigger Q.…”

“…For NTT and iNTT, [49] characterizes various NTT implementations, including the high-radix approach in this paper, and suggests on-the-fly twiddle factor generation. Another approach [10] is to exploit Discrete Galois Transform (DGT)…”

Over 100x Faster Bootstrapping in Fully Homomorphic Encryption through Memory-centric Optimization with GPUs

“…For the CPU implementation of NTT, we use the same approach as in the RNS operation, where each thread takes N residues (i.e., a (i) for a given i) at a time, and perform N -point NTT. For the GPU implementation, we use the hierarchical NTT implementation [KJPA20], which heavily exploits shared memory in GPUs while adopting an earlier approach in [GLD + 08]. Specifically, for every (i)NTT with N residues, we use 8 per-thread (i)NTT kernels, as described in [KJPA20], where each thread in a kernel loads eight residues into the registers at a time.…”

“…For the GPU implementation, we use the hierarchical NTT implementation [KJPA20], which heavily exploits shared memory in GPUs while adopting an earlier approach in [GLD + 08]. Specifically, for every (i)NTT with N residues, we use 8 per-thread (i)NTT kernels, as described in [KJPA20], where each thread in a kernel loads eight residues into the registers at a time. We launch kernels each performing radix-256 or radix-512 (i)NTT, where radix-k divides an N -point transformation into k interleaved N/k-point transformations.…”

“…Our baseline (i)NTT implementation [KJPA20] launches two GPU kernels, each of which performs radix-√ N (i)NTT, resulting in 2N log N/ log √ N accesses for the input and output. Rather than Barrett's algorithm [Bar87], we adopt Shoup's ModMul, as implemented in [KJPA20], which is commonly used for (i)NTT to reduce the operational complexity of ModMuls [CLP17,HS14]. Using Shoup's method adds extra N memory accesses as it demands a precomputed value for each ModMul.…”