DP-VAE: Human-Readable Text Anonymization for Online Reviews with Differentially Private Variational Autoencoders

Abstract

While vast amounts of personal data are shared daily on public online platforms and used by companies and analysts to gain valuable insights, privacy concerns are also on the rise: Modern authorship attribution techniques have proven effective at identifying individuals from their data, such as their writing style or behavior of picking and judging movies. It is hence crucial to develop data sanitization methods that allow sharing of users' data while protecting their privacy and preserving quality and content of the original data.In this paper, we tackle anonymization of textual data and propose an end-to-end differentially private variational autoencoder architecture. Unlike previous approaches that achieve differential privacy on a per-word level through individual perturbations, our solution works at an abstract level by perturbing the latent vectors that provide a global summary of the input texts. Decoding an obfuscated latent vector thus not only allows our model to produce coherent, high-quality output text that is human-readable, but also results in strong anonymization due to the diversity of the produced data. We evaluate our approach on IMDb movie and Yelp business reviews, confirming its anonymization capabilities and preservation of the semantics and utility of the original sentences.

Hash the Universe: Differentially Private Text Extraction with Feature Hashing

“…Additionally, since multiple features can be created for each of the 𝑘 occurrences of a term, we define two data sets as being adjacent if they differ by all 𝐾 features associated with a given term, |𝑥| − |𝑥 ′ | = 𝐾, 𝐾 ≥ 𝑘. Three previous works have explored providing actual user-level differential privacy against text-based linkage attacks 24,25,26 , also known as authorship attribution attacks. At first glance this may sound similar to our work here, however there is key difference: these works focus on protecting the privacy of the people providing the text, rather than protecting the confidentiality of the text itself.…”

The Limits of Word Level Differential Privacy

“…To generate human-readable text, Bo et al (2021) employ an encoder-decoder model similar to ours, but without paraphrasing, and sample output words using (a two-set variant of) the Exponential mechanism (McSherry and Talwar, 2007). Weggenmann et al (2022) propose a differentially private variation of the variational autoencoder and use it as a sequenceto-sequence architecture for text anonymization.…”

Assessing Differentially Private Variational Autoencoders under Membership Inference

“…We furthermore use a domain independent LDP mechanism specifically for VAE, to which we refer as VAE-LDP. VAE-LDP by Weggenmann et al [37] allows a data scientist to use VAE as LDP mechanism to perturb data. This is achieved by limiting the encoders mean and adding noise to the encoders standard deviation before sampling the latent code z during training.…”