Privacy-preserving Neural Representations of Text

Coavoux, Maximin; Narayan, Shashi; Cohen, Shay B.

doi:10.18653/v1/d18-1001

Cited by 92 publications

(140 citation statements)

References 16 publications

(11 reference statements)

Supporting

Mentioning

138

Contrasting

Order By: Relevance

“…The notion of similarity is controlled by a parameter ε ≥ 0 that defines the strength of the privacy guarantee (with ε = 0 representing absolute privacy, and ε = ∞ representing null privacy). Even though DP has been applied to domains such as geolocation [4], social networks [40] and deep learning [1,50], less attention has been paid to adapting variants of DP to the context of Natural Language Processing (NLP) and the text domain [15,59].…”

Section: Introductionmentioning

confidence: 99%

Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations

Feyisetan

Balle

Drake

et al. 2020

Proceedings of the 13th International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Accurately learning from user data while providing quantifiable privacy guarantees provides an opportunity to build better ML models while maintaining user trust. This paper presents a formal approach to carrying out privacy preserving text perturbation using the notion of d χ -privacy designed to achieve geoindistinguishability in location data. Our approach applies carefully calibrated noise to vector representation of words in a high dimension space as defined by word embedding models. We present a privacy proof that satisfies d χ -privacy where the privacy parameter ε provides guarantees with respect to a distance metric defined by the word embedding space. We demonstrate how ε can be selected by analyzing plausible deniability statistics backed up by large scale analysis on G V and T embeddings. We conduct privacy audit experiments against 2 baseline models and utility experiments on 3 datasets to demonstrate the tradeoff between privacy and utility for varying values of ε on different task types. Our results demonstrate practical utility (< 2% utility loss for training binary classifiers) while providing better privacy guarantees than baseline models.

show abstract

Section: Introductionmentioning

confidence: 99%

Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations

Feyisetan

Balle

Drake

et al. 2020

Proceedings of the 13th International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

show abstract

“…Our encoder-decoder architecture together with independence assumptions made in the probabilistic model which decomposes a derivation score in several subtasks can be seen as auxiliary tasks as in (Coavoux et al, 2018).…”

Section: Discussion and Related Workmentioning

confidence: 99%

Representation Learning and Dynamic Programming for Arc-Hybrid Parsing

Roux¹,

Rozenknop²,

Lacroix³

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

We present a new method for transition-based parsing where a solution is a pair made of a dependency tree and a derivation graph describing the construction of the former. From this representation we are able to derive an efficient parsing algorithm and design a neural network that learns vertex representations and arc scores. Experimentally, although we only train via local classifiers, our approach improves over previous arc-hybrid systems and reach state-of-the-art parsing accuracy.

show abstract

“…For comparison, consider the distance between b 1 and b 2 to a third document, b 3 := Chef 1 , breaks 1 , cooking 1 , record 1 . Using the same word embedding metric, 10 we find that…”

Section: Word Embeddingsmentioning

confidence: 97%

“…All of our results apply equally well had we left stopwords in place. 10 We use the same word2vec-based metric as per our experiments; this is described in §6.…”

Section: Word Embeddingsmentioning

confidence: 99%

Generalised Differential Privacy for Text Document Processing

Fernandes

Dras

McIver

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We address the problem of how to "obfuscate" texts by removing stylistic clues which can identify authorship, whilst preserving (as much as possible) the content of the text. In this paper we combine ideas from "generalised differential privacy" and machine learning techniques for text processing to model privacy for text documents. We define a privacy mechanism that operates at the level of text documents represented as "bags-of-words" -these representations are typical in machine learning and contain sufficient information to carry out many kinds of classification tasks including topic identification and authorship attribution (of the original documents). We show that our mechanism satisfies privacy with respect to a metric for semantic similarity, thereby providing a balance between utility, defined by the semantic content of texts, with the obfuscation of stylistic clues. We demonstrate our implementation on a "fan fiction" dataset, confirming that it is indeed possible to disguise writing style effectively whilst preserving enough information and variation for accurate content classification tasks.

show abstract

Privacy-preserving Neural Representations of Text

Cited by 92 publications

References 16 publications

Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations

Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations

Representation Learning and Dynamic Programming for Arc-Hybrid Parsing

Generalised Differential Privacy for Text Document Processing

Contact Info

Product

Resources

About