Privacy-Preserving Deep Sequential Model with Matrix Homomorphic Encryption

Jang, Jaehee; Lee, Younho; Kim, Andrey; Na, Byunggook; Yhee, Donggeon; Lee, Byounghan; Cheon, Jung Hee; Yoon, Sungroh

doi:10.1145/3488932.3523253

Cited by 8 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jang et al [76] propose a variation of the CKKS scheme, MatHEAAN (Matrix HEEAN), that specializes in matrix operations. Based on this scheme the authors Gated Recurrent Units (GRU) [77], to handle sequential data.…”

Section: Improved Computationmentioning

confidence: 99%

“…Jang et al [76], implement GRUs on encrypted data using their proposed encryption scheme MatHEAAN. The internal structure of a GRU is similar to that of an LSTM and introduces high multiplicative depth.…”

Section: Recurrent Layersmentioning

confidence: 99%

“…RNN architectures are not well suited for execution over homomorphically encrypted data [83]; they require either fundamental changes as seen in SHE [65] and RNN Blocks [83], bootstrapping [76], or establishing client communication as seen in [84] and [82]. However, interactive communication takes away some HE key advantages over Secure Multiparty Computation, such as the lower communication overhead and the ability to perform the computation independently from the client.…”

Section: Recurrent Layersmentioning

confidence: 99%

See 2 more Smart Citations

A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning With Fully Homomorphic Encryption

Takabi

Rafiei

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Outsourced computation for neural networks allows users access to state-of-the-art models without investing in specialized hardware and know-how. The problem is that the users lose control over potentially privacy-sensitive data. With homomorphic encryption (HE), a third party can perform computation on encrypted data without revealing its content. In this paper, we reviewed scientific articles and publications in the particular area of Deep Learning Architectures for Privacy-Preserving Machine Learning (PPML) with Fully HE. We analyzed the changes to neural network models and architectures to make them compatible with HE and how these changes impact performance. Next, we find numerous challenges to HEbased privacy-preserving deep learning, such as computational overhead, usability, and limitations posed by the encryption schemes. Furthermore, we discuss potential solutions to the HE PPML challenges. Finally, we propose evaluation metrics that allow for a better and more meaningful comparison of PPML solutions.

show abstract

Section: Improved Computationmentioning

confidence: 99%

Section: Recurrent Layersmentioning

confidence: 99%

Section: Recurrent Layersmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning With Fully Homomorphic Encryption

Takabi

Rafiei

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…And Feng et al propose a hybrid approach that combines HE and garbled circuits for evaluating GRU networks on text analysis tasks [37]. More recently, Jang et al have extended the CKKS-HE scheme to the multi-variate ring learning with errors (RLWE) problem in order to support efficient matrix operations that are required for evaluating GRUs on sequence modeling, regression, and classification tasks [58]. Similar to these works, RHODE protects the confidentiality of the model and the client's evaluation data during the prediction phase.…”

Section: Privacy-preserving Prediction On Machine Learning Modelsmentioning

confidence: 99%

Privacy-Preserving Federated Recurrent Neural Networks

Sav¹,

Abdulrahman²,

Pyrgelis³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present RHODE, a novel system that enables privacy-preserving training of and prediction on Recurrent Neural Networks (RNNs) in a federated learning setting by relying on multiparty homomorphic encryption (MHE). RHODE preserves the confidentiality of the training data, the model, and the prediction data; and it mitigates the federated learning attacks that target the gradients under a passive-adversary threat model. We propose a novel packing scheme, multi-dimensional packing, for a better utilization of Single Instruction, Multiple Data (SIMD) operations under encryption. With multi-dimensional packing, RHODE enables the efficient processing, in parallel, of a batch of samples. To avoid the exploding gradients problem, we also provide several clip-by-value approximations for enabling gradient clipping under encryption. We experimentally show that the model performance with RHODE remains similar to non-secure solutions both for homogeneous and heterogeneous data distribution among the data holders. Our experimental evaluation shows that RHODE scales linearly with the number of data holders and the number of timesteps, sub-linearly and sub-quadratically with the number of features and the number of hidden units of RNNs, respectively. To the best of our knowledge, RHODE is the first system that provides the building blocks for the training of RNNs and its variants, under encryption in a federated learning setting.

show abstract

“…Jang et al [76] introduced an extended CKKS scheme called MatHEAAN that provides efficient matrix representations and operations and improved noise control. The scheme is specifically designed to work with recurrent neural networks.…”

Section: Projections In Complex Applicationsmentioning

confidence: 99%

HEProfiler: An In-Depth Profiler of Approximate Homomorphic Encryption Libraries

Takeshita

Koirala

McKechney

et al. 2022

Preprint

View full text Add to dashboard Cite

Fully Homomorphic Encryption (FHE) allows computation on encrypted data. Various software libraries have implemented the approximate-arithmetic FHE scheme CKKS, which is highly useful for applications in machine learning and data analytics; each of these libraries have differing performance and features. It is useful for developers and researchers to learn details about these libraries' performance and their differences. Some previous work has profiled FHE and CKKS implementations for this purpose, but these comparisons are limited in their fairness and completeness. In this article, we compare four major libraries supporting the CKKS scheme. Working with the maintainers of each of the PALISADE, Microsoft SEAL, HElib, and HEAAN libraries, we devise methods for fair comparisons of these libraries, even with their widely varied development strategies and library architectures. To show the practical performance of these libraries, we present HEProfiler, a simple and extensible framework for profiling C++ FHE libraries. Our experimental evaluation is complete in both the scope of tasks tested and metrics evaluated, allowing us to draw conclusions about the behaviors of different libraries under a wide range of real-world workloads. This is the first work giving experimental comparisons of different bootstrapping-capable CKKS libraries.

show abstract

Privacy-Preserving Deep Sequential Model with Matrix Homomorphic Encryption

Cited by 8 publications

References 13 publications

A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning With Fully Homomorphic Encryption

A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning With Fully Homomorphic Encryption

Privacy-Preserving Federated Recurrent Neural Networks

HEProfiler: An In-Depth Profiler of Approximate Homomorphic Encryption Libraries

Contact Info

Product

Resources

About