Building a RAPPOR with the Unknown: Privacy-Preserving Learning of Associations and Data Dictionaries

Abstract

Techniques based on randomized response enable the collection of potentially sensitive data from clients in a privacy-preserving manner with strong local differential privacy guarantees. A recent such technology, RAPPOR [12], enables estimation of the marginal frequencies of a set of strings via privacy-preserving crowdsourcing. However, this original estimation process relies on a known dictionary of possible strings; in practice, this dictionary can be extremely large and/or unknown. In this paper, we propose a novel decoding algorithm for the RAPPOR mechanism that enables the estimation of "unknown unknowns," i.e., strings we do not know we should be estimating. To enable learning without explicit dictionary knowledge, we develop methodology for estimating the joint distribution of multiple variables collected with RAPPOR. Our contributions are not RAPPOR-specific, and can be generalized to other local differential privacy mechanisms for learning distributions of string-valued random variables.

De-Identification Technique for Iot Wireless Sensor Network Privacy Protection

“…As there are no encryption or decryption involved, there is no risk of leaking a key to an advisory. There're also following up works on RAPPOR to eliminate the need of having to build a dictionary first before data transmission and decoding [15].…”

High-Dimensional Crowdsourced Data Distribution Estimation with Local Privacy

“…In addition, local privacy from the end user can ensure the consistency of the privacy guarantees when there are multiple accesses to users' data, in contrast to non-local privacy schemes that has to properly split and assign privacy budgets to different steps [5], [21], [35]. In existing work [15][12] [14], local privacy is implemented with randomized response technique [34]. However, the correlations and sparsity in high-dimensional data are not well considered, which will cause low scalability and utility for highdimensional data [25], [35].…”

“…However, it is not directly applicable to multiple dimensional data with large domain size since the binary strings will have exponential length increments in terms of the number of dimensions. To address this problem, Fanti et al [14] propose an association learning scheme, which extends the 1-dimensional RAPPOR to estimate the 2-dimensional joint distribution. However, the sparsity in the multi-dimensional domain and the way it iteratively scans RAPPOR strings means that it will incur considerable computational complexity.…”

“…We then propose multi-dimensional joint distribution estimation schemes to obtain both the joint and marginal probability distribution on multidimensional data. Inspired by [14], we first extend the EM-based approach for high-dimensional However, such a straightforward extension does not consider the sparsity in high-dimensional data, which will lead to high complexity for distribution estimation. To guarantee fast estimation, we then present a Lasso-based approach with the cost of slight accuracy degradation.…”

“…For example, an EM-based estimation algorithm [14] was proposed to estimate 2-dimensional joint distribution. However, due to high complexity and overheads, it is only preferable to low dimensions with small domain, which is impractical to many real-world datasets with high dimensions.…”