Logistic regression model training based on the approximate homomorphic encryption

Abstract

BackgroundSecurity concerns have been raised since big data became a prominent tool in data analysis. For instance, many machine learning algorithms aim to generate prediction models using training data which contain sensitive information about individuals. Cryptography community is considering secure computation as a solution for privacy protection. In particular, practical requirements have triggered research on the efficiency of cryptographic primitives.MethodsThis paper presents a method to train a logistic regression model without information leakage. We apply the homomorphic encryption scheme of Cheon et al. (ASIACRYPT 2017) for an efficient arithmetic over real numbers, and devise a new encoding method to reduce storage of encrypted database. In addition, we adapt Nesterov’s accelerated gradient method to reduce the number of iterations as well as the computational cost while maintaining the quality of an output classifier.ResultsOur method shows a state-of-the-art performance of homomorphic encryption system in a real-world application. The submission based on this work was selected as the best solution of Track 3 at iDASH privacy and security competition 2017. For example, it took about six minutes to obtain a logistic regression model given the dataset consisting of 1579 samples, each of which has 18 features with a binary outcome variable.ConclusionsWe present a practical solution for outsourcing analysis tools such as logistic regression analysis while preserving the data confidentiality.Electronic supplementary materialThe online version of this article (10.1186/s12920-018-0401-7) contains supplementary material, which is available to authorized users.

Logistic Regression on Homomorphic Encrypted Data at Scale

“…HE-Based Approaches. Graepel et al (Graepel, Lauter, and Naehrig 2012) presented a homomorphic evaluation algorithm of two binary classifiers (i.e., linear means and Fisher's linear discriminant classifiers), and Kim et al (Kim et al 2018b;2018a) proposed a homomorphic evaluation of logistic regression. However, they provided only a proofof-concept evaluation, where small-scale training datasets (consisting of only dozens of samples and features) are considered.…”

“…In addition to the sheer amount of computation, the use of various complex operations, such as floating-point arithmetic and non-polynomial functions (e.g., sigmoid), makes it challenging to apply HE to machine learning algorithms. Indeed, HEs have been applied to machine learning algorithms only in non-realistic settings (Graepel, Lauter, and Naehrig 2012;Kim et al 2018a) where only small-size training data over a small number of features are considered; or, they have been applied only on the prediction phase (Gilad-Bachrach et al 2016;Li et al 2017;Juvekar, Vaikuntanathan, and Chandrakasan 2018;Bourse et al 2017) where the amount of computation is much smaller than that of the training phase.…”

High Performance Logistic Regression for Privacy-Preserving Genome Analysis

“…One scenario that was considered in previous works [4,5,6] is the setting in which a data owner holds the data while another party (the data processor), such as a cloud service, is responsible for the model training. These solutions usually rely on homomorphic encryption, with the data owner encrypting and sending their data to the data processor who performs computations on the encrypted data without having to decrypt it.…”

Unsupervised Machine Learning on Encrypted Data

“…Machine Learning as an application of FHE was first proposed in [35], and subsequently there have been numerous works on the subject, to our knowledge all concerned with supervised learning. The most popular of these applications seem to be (Deep) Neural Networks (see [26], [21], [10], [36], and [7]) and (Linear) Regression (e.g., [32], [17], [4] or [22]), though there is also some work on other algorithm classes like decision trees and random forests ( [41]), or logistic regression ( [6], [30], [29] and [5]). In contrast, our work is concerned with the clustering problem from unsupervised Machine Learning.…”