Privacy accounting and quality control in the sage differentially private ML platform

Abstract

Companies increasingly expose machine learning (ML) models trained over sensitive user data to untrusted domains, such as end-user devices and wide-access model stores. This creates a need to control the data's leakage through these models. We present Sage, a differentially private (DP) ML platform that bounds the cumulative leakage of training data through models. Sage builds upon the rich literature on DP ML algorithms and contributes pragmatic solutions to two of the most pressing systems challenges of global DP: running out of privacy budget and the privacy-utility tradeoff. To address the former, we develop block composition, a new privacy loss accounting method that leverages the growing database regime of ML workloads to keep training models endlessly on a sensitive data stream while enforcing a global DP guarantee for the stream. To address the latter, we develop privacyadaptive training, a process that trains a model on growing amounts of data and/or with increasing privacy parameters until, with high probability, the model meets developerconfigured quality criteria. Sage's methods are designed to integrate with TensorFlow-Extended, Google's open-source ML platform. They illustrate how a systems focus on characteristics of ML workloads enables pragmatic solutions that are not apparent when one focuses on individual algorithms, as most DP ML literature does.

Differential Privacy Preservation in Robust Continual Learning

“…Some of the most prominent demands, which are close in spirit to the requirements of CL, as we need here, are listed as follows: R1: Endless execution R2: Multiple usage of data subsets R3: Capability of changing DP parameters during the execution Satisfying all the R1-R3 together is hard, therefore related papers address only one or two of these requirements. Along this line, two recent DL-based papers of [23], [24] have enabled DP to work on growing databases (dynamic datasets). More explicitly, to address R1 Cummings et al have considered a scheduler to re-execute the DL algorithms whenever the new received data is sufficient [24].…”

“…Note that, since each block of data might be used by different DL algorithms corresponding to the pipelines, calculating the PB spent by the whole pipelines would be challenging. To reach this goal, the authors of [23], have proposed the so-called block composition theorem by which the DL algorithms are executed till the PB consumption of each block 1 does not exceed the predefined GPB. To achieve the desired accuracy, with the aim of re-training the pipelines, either the relevant PB of each pipeline or the number of available samples is doubled.…”

“…The everlasting approach of CL is a serious impediment to deploy either of the proposed solutions in [23] or [24]. In particular, if one intends to add DP to CL, the limited GPB hinders the process to be continued.…”

“…A serious impediment to deploy either of the proposed solutions in [23] or [24] in a CL process, is the data coming from the stream as well as samples stored in the EM to avoid catastrophic forgetting (CF). Note that, although the learners observe most of the data coming from the stream just once, a small portion stored in the EM is observed several times.…”

“…where 𝑃𝐵 F $ ,D % stands for the consumed PB of 𝑖th iteration of the learner 𝑙 $ . By doing so, we can calculate the total consumed PB for the sample via feeding 𝑃𝐵 E $ to the Block Composition Theorem (BCT) [23]. Tracking the behavior of 𝑃𝐵 E $ , if it exceeds the GPB, we no longer use that sample in our CL procedure.…”