Shuffle Private Linear Contextual Bandits

Abstract

Differential privacy (DP) has been recently introduced to linear contextual bandits to formally address the privacy concerns in its associated personalized services to participating users (e.g., recommendations). Prior work largely focus on two trust models of DP -the central model, where a central server is responsible for protecting users' sensitive data, and the (stronger) local model, where information needs to be protected directly on users' side. However, there remains a fundamental gap in the utility achieved by learning algorithms under these two privacy models, e.g., O( √ T ) regret in the central model as compared to O(T 3/4 ) regret in the local model, if all users are unique within a learning horizon T . In this work, we aim to achieve a stronger model of trust than the central model, while suffering a smaller regret than the local model by considering recently popular shuffle model of privacy. We propose a general algorithmic framework for linear contextual bandits under the shuffle trust model, where there exists a trusted shuffler -in between users and the central server-that randomly permutes a batch of users data before sending those to the server. We then instantiate this framework with two specific shuffle protocols -one relying on privacy amplification of local mechanisms, and another incorporating a protocol for summing vectors and matrices of bounded norms. We prove that both these instantiations lead to regret guarantees that significantly improve on that of the local model, and can potentially be of the order O(T 3/5 ) if all users are unique. We also verify this regret behavior with simulations on synthetic data. Finally, under the practical scenario of non-unique users, we show that the regret of our shuffle private algorithm scale as O(T 2/3 ), which matches that the central model could achieve in this case.

Concurrent Shuffle Differential Privacy Under Continual Observation

“…, which clearly has optimal asymptotic dependence on n even without privacy. This improves over the result of Chowdhury & Zhou (2022b) who obtained regret of Õ(n 3/5 / √ ε) in the sequential shuffle model. That is, we reduce the regret to optimal, from Õ(n 3/5 / √ ε) to Õ( √ n/ √ ε), while essentially maintaining the same trust model (small number of concurrent shufflers instead of just one).…”

“…That is, we reduce the regret to optimal, from Õ(n 3/5 / √ ε) to Õ( √ n/ √ ε), while essentially maintaining the same trust model (small number of concurrent shufflers instead of just one). This answers positively the open question of Chowdhury & Zhou (2022b).…”

“…4 Several works have studied the private multiarmed bandit problem (Mishra & Thakurta, 2015;Tossou & Dimitrakakis, 2017;Sajed & Sheffet, 2019;Ren et al, 2020a;Chen et al, 2020;Zhou & Tan, 2021;Dubey, 2021), the private contextual linear bandit problem (Shariff & Sheffet, 2018;Zheng et al, 2020;Han et al, 2020;Ren et al, 2020b;Garcelon et al, 2022), and the more general private reinforcement learning (Vietri et al, 2020;Garcelon et al, 2021;Chowdhury & Zhou, 2022a) problem, in both local and centralized models of privacy. The regret gap between the two models (when the contexts are arbitrary, not stochastic (Han et al, 2021)) has shrunk using the intermediate sequential shuffle model (Tenenbaum et al, 2021;Chowdhury & Zhou, 2022b;Garcelon et al, 2022). See Section 5 for further discussion of these results for private contextual linear bandits.…”

“…To adapt the shuffle model to adaptive algorithms (e.g., bandits, sum estimates etc.) under continual observation, Tenenbaum et al (2021); Cheu et al (2022) and Chowdhury & Zhou (2022b) divide the users into continuous batches, and run a shuffle-DP (SDP) mechanism over each batch separately. When a new batch starts, the server selects the next shuffle mechanism (encoder and size), possibly as a function of the outputs of the previous shuffle mechanisms (i.e., it may be adaptive).…”

“…Recall that the private data of the t'th user is its value b t . In the sequential shuffle model (Tenenbaum et al, 2021;Cheu et al, 2022;Chowdhury & Zhou, 2022b), since each user participates in exactly one shuffle mechanism, we ensure (ε, δ) differential privacy of the entire algorithm by making each executed shuffle mechanism (ε, δ)-SDP.…”