Obfuscation for Privacy-preserving Syntactic Parsing

Abstract

The goal of homomorphic encryption is to encrypt data such that another party can operate on it without being explicitly exposed to the content of the original data. We introduce an idea for a privacy-preserving transformation on natural language data, inspired by homomorphic encryption. Our primary tool is obfuscation, relying on the properties of natural language. Specifically, a given English text is obfuscated using a neural model that aims to preserve the syntactic relationships of the original sentence so that the obfuscated sentence can be parsed instead of the original one. The model works at the word level, and learns to obfuscate each word separately by changing it into a new word that has a similar syntactic role. The text obfuscated by our model leads to better performance on three syntactic parsers (two dependency and one constituency parsers) in comparison to an upper-bound random substitution baseline. More specifically, the results demonstrate that as more terms are obfuscated (by their part of speech), the substitution upper bound significantly degrades, while the neural model maintains a relatively high performing parser. All of this is done without much sacrifice of privacy compared to the random substitution upper bound. We also further analyze the results, and discover that the substituted words have similar syntactic properties, but different semantic content, compared to the original words.

A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples

“…This finding is quite close in spirit to our conclusion in Section 4.5. Hu et al (2020) also put forth efforts to modify the text in syntactic tasks while preserving the original syntactic structure. However, their goal is to preserve privacy via the modification of words that could disclose sensitive information.…”

TextHide: Tackling Data Privacy in Language Understanding Tasks

“…To deal with explicit privacy leakage in NLP, Zhang et al (2018b) added DP noise to TF-IDF (Salton andMcGill, 1986) textual vectors, andHu et al (2020) obfuscated the text by substituting each word with a new word of similar syntactic role. However, both approaches suffer large utility loss when trying to ensure practical privacy.…”

“…Adversarial learning Hu et al, 2020) has been used to address implicit leakage to learn representations that are invariant to private-sensitive attributes. Similarly, Mosallanezhad et al (2019) used reinforcement learning to automatically learn a strategy to reduce privateattribute leakage by playing against an attributeinference attacker.…”