Toward Distribution Estimation under Local Differential Privacy with Small Samples

With the advent of the era of big data, privacy issues have been becoming a hot topic in public. Local differential privacy (LDP) is a state-of-the-art privacy preservation technique that allows to perform big data analysis (e.g., statistical estimation, statistical learning, and data mining) while guaranteeing each individual participant’s privacy. In this paper, we present a comprehensive survey of LDP. We first give an overview on the fundamental knowledge of LDP and its frameworks. We then introduce the mainstream privatization mechanisms and methods in detail from the perspective of frequency oracle and give insights into recent studied on private basic statistical estimation (e.g., frequency estimation and mean estimation) and complex statistical estimation (e.g., multivariate distribution estimation and private estimation over complex data) under LDP. Furthermore, we present current research circumstances on LDP including the private statistical learning/inferencing, private statistical data analysis, privacy amplification techniques for LDP, and some application fields under LDP. Finally, we identify future research directions and open challenges for LDP. This survey can serve as a good reference source for the research of LDP to deal with various privacy-related scenarios to be encountered in practice.

show abstract

“…e schemes are suitable for distribution estimation with small samples while guaranteeing LDP. Murakami et al [59] studied on the locally…”

Section: Distribution Estimation With Small Sample Problemmentioning

confidence: 99%

A Comprehensive Survey on Local Differential Privacy

Xiong

Liu

et al. 2020

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Various data analytics and machine learning problems have been studied under LDP, such as probability distribution estimation [15], [23], [30], [32], [42], heavy hitter discovery [4], [7], [33], [39], frequent new term discovery [37], frequency estimation [5], [38], frequent itemset mining [40], marginal release [10], clustering [31], and hypothesis testing [20].…”

Section: Related Workmentioning

confidence: 99%

Collecting and Analyzing Multidimensional Data with Local Differential Privacy

Wang

Xiao

Yang

et al. 2019

2019 IEEE 35th International Conference on Data Engineering (ICDE)

244

271

View full text Add to dashboard Cite

Local differential privacy (LDP) is a recently proposed privacy standard for collecting and analyzing data, which has been used, e.g., in the Chrome browser, iOS and macOS. In LDP, each user perturbs her information locally, and only sends the randomized version to an aggregator who performs analyses, which protects both the users and the aggregator against private information leaks. Although LDP has attracted much research attention in recent years, the majority of existing work focuses on applying LDP to complex data and/or analysis tasks. In this paper, we point out that the fundamental problem of collecting multidimensional data under LDP has not been addressed sufficiently, and there remains much room for improvement even for basic tasks such as computing the mean value over a single numeric attribute under LDP. Motivated by this, we first propose novel LDP mechanisms for collecting a numeric attribute, whose accuracy is at least no worse (and usually better) than existing solutions in terms of worst-case noise variance. Then, we extend these mechanisms to multidimensional data that can contain both numeric and categorical attributes, where our mechanisms always outperform existing solutions regarding worst-case noise variance. As a case study, we apply our solutions to build an LDP-compliant stochastic gradient descent algorithm (SGD), which powers many important machine learning tasks. Experiments using real datasets confirm the effectiveness of our methods, and their advantages over existing solutions.

show abstract

“…Recently, more works propose to apply DP or LDP in data mining and machine learning applications, such as clustering [42], Bayesian inference [43], frequent itemset mining [44] and probability distribution estimation [44], [45], [46], [47]. However, only a few recent works aim to use LDP in deep learning.…”

Section: Related Workmentioning

confidence: 99%

Locally Differentially Private Distributed Deep Learning via Knowledge Distillation

Zhuang¹,

Li²,

Chang³

2022

Preprint

View full text Add to dashboard Cite

Deep learning often requires a large amount of data. In real-world applications, e.g., healthcare applications, the data collected by a single organization (e.g., hospital) is often limited, and the majority of massive and diverse data is often segregated across multiple organizations. As such, it motivates the researchers to conduct distributed deep learning, where the data user would like to build DL models using the data segregated across multiple different data owners. However, this could lead to severe privacy concerns due to the sensitive nature of the data, thus the data owners would be hesitant and reluctant to participate. We propose LDP-DL, a privacy-preserving distributed deep learning framework via local differential privacy and knowledge distillation, where each data owner learns a teacher model using its own (local) private dataset, and the data user learns a student model to mimic the output of the ensemble of the teacher models. In the experimental evaluation, a comprehensive comparison has been made among our proposed approach (i.e., LDP-DL), DP-SGD, PATE and DP-FL, using three popular deep learning benchmark datasets (i.e., CIFAR10, MNIST and FashionMNIST). The experimental results show that LDP-DL consistently outperforms the other competitors in terms of privacy budget and model accuracy.

show abstract

Toward Distribution Estimation under Local Differential Privacy with Small Samples

Cited by 29 publications

References 45 publications

A Comprehensive Survey on Local Differential Privacy

A Comprehensive Survey on Local Differential Privacy

Collecting and Analyzing Multidimensional Data with Local Differential Privacy

Locally Differentially Private Distributed Deep Learning via Knowledge Distillation

Contact Info

Product

Resources

About