Constant matters: Fine-grained Complexity of Differentially Private Continual Observation

Abstract

We study fine-grained error bounds for differentially private algorithms for averaging and counting in the continual observation model. For this, we use the completely bounded spectral norm (cb norm) from operator algebra. For a matrix W, its cb norm is defined aswhere Q • W denotes the Schur product and • denotes the spectral norm. We bound the cb norm of two fundamental matrices studied in differential privacy under the continual observation model: the counting matrix M counting and the averaging matrix M average . For M counting , we give lower and upper bound whose additive gap is 1 + 1 π . Our factorization also has two desirable properties sufficient for streaming setting: the factorization contains of lower-triangular matrices and the number of distinct entries in the factorization is exactly T. This allows us to compute the factorization on the fly while requiring the curator to store a T-dimensional vector. For M average , we show an additive gap between the lower and upper bound of ≈ 0.64.

Almost Tight Error Bounds on Differentially Private Continual Counting

“…On the other hand, given its application in real-world deployments mentioned above, designing an algorithm for continual counting with provable mean-squared error and one with smallest constant is highly desirable. The importance of having small constants was also recently pointed out by Fichtenberger, Henzinger, and Upadhyay [FHU22] in the continual observation model. This question was also the center of a subsequent work by Asi, Feldman, and Talwar [AFT22] on mean estimation in the local model of privacy.…”

“…where L 2→∞ is the maximum of the 2-norm of the rows of L. The • cb norm plays an important role in bounding the ∞ -error [FHU22]. It has been extensively studied in operator algebra and tight bounds are known for M count cb [Mat93].…”

“…Depending on the downstream use case, the performance of a differentially private continual mechanism is either measured in terms of absolute error (aka ∞ -error) or mean squared error (aka 2 2 -error) over the different time steps (defined below). For continual counting, Fichtenberger, Henzinger, and Upadhyay [FHU22] gave an efficient algorithm based on a subclass of matrix mechanism known as factorization mechanism and showed that its ∞ -error is almost tight for any matrix mechanism, not only in the asymptotic setting but even with almost matching constants for the upper and lower bounds. Concurrently to [FHU22], Denisov, McMahan, Rush, Smith, and Thakurta [DMR + 22] studied the 2 2 error for continual counting and gave conditions that a factorization has to fulfill to give an optimal 2 2 -error.…”

“…For continual counting, Fichtenberger, Henzinger, and Upadhyay [FHU22] gave an efficient algorithm based on a subclass of matrix mechanism known as factorization mechanism and showed that its ∞ -error is almost tight for any matrix mechanism, not only in the asymptotic setting but even with almost matching constants for the upper and lower bounds. Concurrently to [FHU22], Denisov, McMahan, Rush, Smith, and Thakurta [DMR + 22] studied the 2 2 error for continual counting and gave conditions that a factorization has to fulfill to give an optimal 2 2 -error. They also proposed the use of a fixed point algorithms to compute the factorization, but they do not give an explicit factorization or any provable 2 2 -error bound of their mechanism.…”

Almost Tight Error Bounds on Differentially Private Continual Counting

“…The setting in [CSS11,DNPR10] was generalized in an elegant work by Bolot et al [BFM + 13], who studied differentially private decaying sums under continual observation for polynomial decay, exponential decay, and the sliding window model. While they gave algorithms with a polylogarithmic additive error bound, they suffer from various limitations which limit their usage in applications, including the two main motivations (estimating infectious disease spread [DNPR10,FHU22] and private online optimization [KMS + 21]) behind the recent interest in private continual observation:…”