FastSecNet: An Efficient Cryptographic Framework for Private Neural Network Inference

Recent Progress and Applications of HfO2-Based Ferroelectric Memory

“…A recent art Cheetah [13] proposes a special encoding method to encode vectors and matrices into HE polynomials, which achieves state-of-the-art performance in computing matrix-vector multiplication and convolutions. Iron [10] realizes that matrix-matrix multiplication (rather than matrix-vector multiplication) dominates in transformer-based inference, and therefore improves the vanilla polynomial encoding by introducing a blocking method that prioritizes the batch dimension. Despite the optimization, some of the non-linear functions (e.g., GELU, softmax and layer normalization layers) are fundamentally expensive in private inference.…”

“…Despite the optimization, some of the non-linear functions (e.g., GELU, softmax and layer normalization layers) are fundamentally expensive in private inference. For instance, Iron [10] reports that running a single inference on BERT-Tiny [3] requires 50 seconds time and 2GB transmission. Two recent studies explore replacing these fundamentally expensive non-linear functions with operators that are more friendly in private inference.…”

“…The core linear protocol for private inference over linear layers is the matrix multiplication protocol. It is realized using homomorphic encryption, with the polynomial encoding primitive first proposed by [13] and extended by [10]. We start with a simple situation where one party holds A and the other party holds B.…”

“…The private inference paradigm of neural networks has recently emerged as a solution to the problem of using private data in model inference [7,16,20,23,13,10,27]. In this paradigm, the client submits an encrypted version of its input and works collaboratively with the service provider to obtain an encrypted inference result that can only be recovered by the client itself.…”

“…The state-of-the-art works [13] and [10] demonstrate the possibility of private inference on popular neural networks (e.g., convolutional networks and transformers) in computer vision and natural language processing, respectively. We observe that the time and communication cost of private inference on LLMs consisting of multiple transformer blocks is much higher than that of the traditional convolutional networks For instance, even on a model as small as BERT-Tiny [3], [10] takes ∼50 seconds and 2GB of communication for a single inference, while [13] could scale to ResNet-32 [11] with 15 seconds and 0.11 GB communication for an inference. This difference is because transformers use sophisticated nonlinear functions that are computationally-unfriendly to the cryptographic primitives.…”