Exponential Separations in Local Differential Privacy

Joseph, Matthew; Mao, Jieming; Roth, Aaron

doi:10.1137/1.9781611975994.31

“…Such protocols, first defined in Kasiviswanathan et al (2011) are referred to as compositional -LDP. They are known to be exponentially weaker than the general interactive LDP protocols although the separation is known only for rather unnatural problems (Joseph et al, 2019b). The converse of this connection is also known: SQ algorithms can be simulated by -compositional LDP protocols (and this simulation preserves the number of rounds of interaction) (Kasiviswanathan et al, 2011).…”

Section: Models Of Computationmentioning

confidence: 99%

“…We hope that the construction in this paper will provide a useful step toward lower bounds against multi-round SQ or LDP algorithms. We remark, however, that general multi-round LDP protocols can be stronger than statistical query algorithms (Joseph et al, 2019b) and thus may require an entirely different approach.…”

Section: Introductionmentioning

confidence: 97%

Interaction is Necessary for Distributed Learning with Privacy or Communication Constraints

Dagan

¹

,

Feldman

²

2021

JPC

View full text Add to dashboard Cite

Local differential privacy (LDP) is a model where users send privatized data to an untrusted central server whose goal it to solve some data analysis task. In the non-interactive version of this model the protocol consists of a single round in which a server sends requests to all users then receives their responses. This version is deployed in industry due to its practical advantages and has attracted significant research interest. Our main result is an exponential lower bound on the number of samples necessary to solve the standard task of learning a large-margin linear separator in the non-interactive LDP model. Via a standard reduction this lower bound implies an exponential lower bound for stochastic convex optimization and specifically, for learning linear models with a convex, Lipschitz and smooth loss. These results answer the questions posed by Smith, Thakurta, and Upadhyay (IEEE Symposium on Security and Privacy 2017) and Daniely and Feldman (NeurIPS 2019). Our lower bound relies on a new technique for constructing pairs of distributions with nearly matching moments but whose supports can be nearly separated by a large margin hyperplane. These lower bounds also hold in the model where communication from each user is limited and follow from a lower bound on learning using non-adaptive statistical queries.

show abstract

“…We hope that the construction in this paper will provide a useful step toward lower bounds against multi-round SQ or LDP algorithms. We remark, however, that general multi-round LDP protocols can be stronger than statistical query algorithms [37] and thus may require an entirely different approach (see discussion in Section 2.1 for more details).…”

Section: Convex Loss Optimization Of Linear Modelsmentioning

confidence: 99%

“…Joseph et al [36,37] explore a different aspect of interactivity in LDP. Specifically, they distinguish between two types of interactive protocols: fully-interactive and sequentially-interactive ones.…”

Section: Related Workmentioning

confidence: 99%

“…Such protocols, first defined in [39] are referred to as compositional ϵ-LDP. They are known to be exponentially weaker than the general interactive LDP protocols although the separation is known only for rather unnatural problems [37]. The converse of this connection is also known: SQ algorithms can be simulated by ϵ-compositional LDP protocols (and this simulation preserves the number of rounds of interaction) [39].…”

Section: Theorem 23 ([39]mentioning

confidence: 99%

See 1 more Smart Citation

Interaction is necessary for distributed learning with privacy or communication constraints

Dagan

¹

,

Feldman

²

2020

Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

Local differential privacy (LDP) is a model where users send privatized data to an untrusted central server whose goal it to solve some data analysis task. In the non-interactive version of this model the protocol consists of a single round in which a server sends requests to all users then receives their responses. This version is deployed in industry due to its practical advantages and has attracted significant research interest. Our main result is an exponential lower bound on the number of samples necessary to solve the standard task of learning a large-margin linear separator in the non-interactive LDP model. Via a standard reduction this lower bound implies an exponential lower bound for stochastic convex optimization and specifically, for learning linear models with a convex, Lipschitz and smooth loss. These results answer the questions posed by Smith, Thakurta, and Upadhyay (IEEE Symposium on Security and Privacy 2017) and Daniely and Feldman (NeurIPS 2019). Our lower bound relies on a new technique for constructing pairs of distributions with nearly matching moments but whose supports can be nearly separated by a large margin hyperplane. These lower bounds also hold in the model where communication from each user is limited and follow from a lower bound on learning using non-adaptive statistical queries.

show abstract

The Role of Interactivity in Local Differential Privacy

Joseph

¹

,

Mao

²

,

Neel

³

et al. 2019

2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS)

Self Cite

View full text Add to dashboard Cite

We study the power of interactivity in local differential privacy. First, we focus on the difference between fully interactive and sequentially interactive protocols. Sequentially interactive protocols may query users adaptively in sequence, but they cannot return to previously queried users. The vast majority of existing lower bounds for local differential privacy apply only to sequentially interactive protocols, and before this paper it was not known whether fully interactive protocols were more powerful.We resolve this question. First, we classify locally private protocols by their compositionality, the multiplicative factor k ≥ 1 by which the sum of a protocol's single-round privacy parameters exceeds its overall privacy guarantee. We then show how to efficiently transform any fully interactive k-compositional protocol into an equivalent sequentially interactive protocol with an O(k) blowup in sample complexity. Next, we show that our reduction is tight by exhibiting a family of problems such that for any k, there is a fully interactive k-compositional protocol which solves the problem, while no sequentially interactive protocol can solve the problem without at least anΩ(k) factor more examples.We then turn our attention to hypothesis testing problems. We show that for a large class of compound hypothesis testing problems -which include all simple hypothesis testing problems as a special case -a simple noninteractive test is optimal among the class of all (possibly fully interactive) tests.

show abstract

Exponential Separations in Local Differential Privacy

Cited by 18 publications

References 24 publications

Interaction is Necessary for Distributed Learning with Privacy or Communication Constraints

Interaction is Necessary for Distributed Learning with Privacy or Communication Constraints

Interaction is necessary for distributed learning with privacy or communication constraints

The Role of Interactivity in Local Differential Privacy

Contact Info

Product

Resources

About