Adaptive Control of Differentially Private Linear Quadratic Systems

Abstract

In this paper we study the problem of regret minimization in reinforcement learning (RL) under differential privacy constraints. This work is motivated by the wide range of RL applications for providing personalized service, where privacy concerns are becoming paramount. In contrast to previous works, we take the first step towards non-tabular RL settings, while providing a rigorous privacy guarantee. In particular, we consider the adaptive control of differentially private linear quadratic (LQ) systems. We develop the first private RL algorithm, Private-OFU-RL which is able to attain a sub-linear regret while guaranteeing privacy protection. More importantly, the additional cost due to privacy is only on the order of ln(1/δ) 1/4 ε 1/2 given privacy parameters ε, δ > 0. Through this process, we also provide a general procedure for adaptive control of LQ systems under changing regularizers, which not only generalizes previous non-private controls, but also serves as the basis for general private controls.

Differentially Private Regret Minimization in Episodic Markov Decision Processes

“…On the other hand, Garcelon et al (2020) design the first private RL algorithm -LDP-OBI -with regret and LDP guarantees. Recently, Chowdhury, Zhou, and Shroff (2021) study linear quadratic regulators under the JDP constraint. It is worth noting that all these prior work consider only value-based RL algorithms, and a study on policy-based private RL algorithms remains elusive.…”

City Water Resources Vulnerability: The Case of Jinan and Qingdao in Shandong Province, China

“…Noting that the information structure in the decentralized control systems mentioned above has the following feature, that is, all or part of historical control inputs of the controllers are shared with the other controllers. However, the case, where the controllers have its own private control inputs, has not been addressed in decentralized control system, which has applications in a personalized healthcare setting, in the states of a virtual keyboard user (e.g., Google GBoard users) and in the social robot for second language education of children [27]. It should be noted that the information structure where the control information are unavailable to the other decision makers will cause the estimation gain depends on the control gain and vice versa, which means the forward and backward Riccati equations are coupled, and make the calculation more complicated.…”