Latent Dirichlet Allocation Model Training With Differential Privacy

Zhao, Fangyuan; Ren, Xuebin; Yang, Shusen; Qing, Han; Zhao, Peng; Yang, Xinyu

doi:10.1109/tifs.2020.3032021

Cited by 20 publications

(32 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A commonly used criterion is perplexity [59]- [61]. In general, the lower the perplexity value, the better the quality of the model [60]. We compute perplexity values while varying the number of topics and adopt the number at which the rate of change in perplexity values becomes low.…”

Section: ) Technological Relevancementioning

confidence: 99%

A Supervised Learning-Based Approach to Anticipating Potential Technology Convergence

2022

View full text Add to dashboard Cite

show abstract

Section: ) Technological Relevancementioning

confidence: 99%

A Supervised Learning-Based Approach to Anticipating Potential Technology Convergence

2022

View full text Add to dashboard Cite

show abstract

“…Zhao et al (Zhao et al, 2019) propose a locally private LDA training algorithm on crowdsourced data to provide local DP for individual data contributors. (Zhao et al, 2020) propose a centralized privacy-preserving algorithm that can prevent data inference from the intermediate statistics in CGS training. Variational Bayes for parameter estimation of LDA is the focus of (Park et al, 2016).…”

Section: Related Workmentioning

confidence: 99%

“…Since counting n t k needs to touch original dataset, n t k is considered as sensitive information and thus needs to be protected. HDP-LDA (Zhao et al, 2020) suggests adding noise, for example, Laplace noise, to each n t k independently in each iteration of CGS. Thus, even the adversary can monitor the whole training process of CGS, n t k in each iteration could be protected.…”

Section: Latent Dirichlet Allocation and Collapsedmentioning

confidence: 99%

“…HDP-LDA, proposed by Zhao et al (Zhao et al, 2020), has been demonstrated to be effective and outperforms other relevant works (Park et al, 2016) (Zhu et al, 2016) (Zhao et al, 2019) when protecting sensitive word-count information in CGS training. HDP-LDA injects noise into word counts in each training iteration.…”

Section: Introductionmentioning

confidence: 95%

“…Differential privacy (DP) is a de-facto standard of privacy protection definition with a rigorous mathematical proof and is widely used for quantifying the privacy risks of random algorithms. To address privacy issues when touching datasets containing sensitive information in the training process of LDA, some works (Park et al, 2016) (Zhu et al, 2016) (Wang et al, 2020) (Zhao et al, 2019) (Zhao et al, 2020) combine DP with LDA. In this study, we focus on LDA training algorithms based on collapsed Gibbs sampling (CGS).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improving Privacy Guarantee and Efficiency of Latent Dirichlet Allocation Model Training Under Differential Privacy

Huang¹,

Chen²

2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Latent Dirichlet allocation (LDA), a widely used topic model, is often employed as a fundamental tool for text analysis in various applications. However, the training process of the LDA model typically requires massive text corpus data. On one hand, such massive data may expose private information in the training data, thereby incurring significant privacy concerns. On the other hand, the efficiency of the LDA model training may be impacted, since LDA training often needs to handle these massive text corpus data. To address the privacy issues in LDA model training, some recent works have combined LDA training algorithms that are based on collapsed Gibbs sampling (CGS) with differential privacy. Nevertheless, these works usually have a high accumulative privacy budget due to vast iterations in CGS. Moreover, these works always have low efficiency due to handling massive text corpus data. To improve the privacy guarantee and efficiency, we combine a subsampling method with CGS and propose a novel LDA training algorithm with differential privacy, SUB-LDA. We find that subsampling in CGS naturally improves efficiency while amplifying privacy. We propose a novel metric, the efficiency-privacy function, to evaluate improvements of the privacy guarantee and efficiency. Based on a conventional subsampling method, we propose an adaptive subsampling method to improve the model's utility produced by SUB-LDA when the subsampling ratio is small. We provide a comprehensive analysis of SUB-LDA, and the experiment results validate its efficiency and privacy guarantee improvements.

show abstract

LoHDP: Adaptive local differential privacy for high‐dimensional data publishing

Shen,

Cai,

Huang

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

SummaryThe increasing availability of high‐dimensional data collected from numerous users has led to the need for multi‐dimensional data publishing methods that protect individual privacy. In this paper, we investigate the use of local differential privacy for such purposes. Existing solutions calculate pairwise attribute marginals to construct probabilistic graphical models for generating attribute clusters. These models are then used to derive low‐dimensional marginals of these clusters, allowing for an approximation of the distribution of the original dataset and the generation of synthetic datasets. Existing solutions have limitations in computing the marginals of pairwise attributes and multi‐dimensional distribution on attribute clusters, as well as constructing relational dependency graphs that contain large clusters. To address these problems, we propose LoHDP, a high‐dimensional data publishing method composed of adaptive marginal computing and an effective attribute clustering method. The adaptive local marginal calculates any k‐dimensional marginals required in the algorithm. In particular, methods such as sampling‐based randomized response are used instead of privacy budget splits to perturb user data. The attribute clustering method measures the correlation between pairwise attributes using an effective method, reduces the search space during the construction of the dependency graph using high‐pass filtering technology, and realizes dimensionality reduction by combining sufficient triangulation operation. We demonstrate through extensive experiments on real datasets that our LoHDP method outperforms existing methods in terms of synthetic dataset quality.

show abstract

Latent Dirichlet Allocation Model Training With Differential Privacy

Cited by 20 publications

References 28 publications

A Supervised Learning-Based Approach to Anticipating Potential Technology Convergence

A Supervised Learning-Based Approach to Anticipating Potential Technology Convergence

Improving Privacy Guarantee and Efficiency of Latent Dirichlet Allocation Model Training Under Differential Privacy

LoHDP: Adaptive local differential privacy for high‐dimensional data publishing

Contact Info

Product

Resources

About