Deception in Optimal Control

Abstract

In this paper, we consider an adversarial scenario where one agent seeks to achieve an objective and its adversary seeks to learn the agent's intentions and prevent the agent from achieving its objective. The agent has an incentive to try to deceive the adversary about its intentions, while at the same time working to achieve its objective. The primary contribution of this paper is to introduce a mathematically rigorous framework for the notion of deception within the context of optimal control. The central notion introduced in the paper is that of a belief-induced reward: a reward dependent not only on the agent's state and action, but also adversary's beliefs. Design of an optimal deceptive strategy then becomes a question of optimal control design on the product of the agent's state space and the adversary's belief space. The proposed framework allows for deception to be defined in an arbitrary control system endowed with a reward function, as well as with additional specifications limiting the agent's control policy. In addition to defining deception, we discuss design of optimally deceptive strategies under uncertainties in agent's knowledge about the adversary's learning process. In the latter part of the paper, we focus on a setting where the agent's behavior is governed by a Markov decision process, and show that the design of optimally deceptive strategies under lack of knowledge about the adversary naturally reduces to previously discussed problems in control design on partially observable or uncertain Markov decision processes. Finally, we present two examples of deceptive strategies: a "cops and robbers" scenario and an example where an agent may use camouflage while moving. We show that optimally deceptive strategies in such examples follow the intuitive idea of how to deceive an adversary in the above settings.

Modelling Strategic Deceptive Planning in Adversarial Multi-Agent Systems

“…Given a known outcome matrix and opponent model, the algorithm selects a communication that maximises the deceiver's outcome for a single action. This algorithm is therefore most similar to deception-maximising approaches [18,19,21,22,26], and is not suitable for modelling strategic deception in OR simulations.…”

“…Recent work on computational deception has focused on 'singular acts of deception' [28], where an agent attempts to maximise the deceptiveness of an action or plan according to a predefined metric [18,19,21,22,26]. These approaches address decision problems for which the deception itself is the goal: success is measured according to the reduced accuracy of the observer's goal recognition model.…”

Information-Guided Temporal Logic Inference with Prior Knowledge

“…4, there are three banks locating in three 1 × 1 blocks in the state space, where the robber's goal is to reach bank 2 while deceiving the cop (as an adversary) to believe that the robber is trying to reach the other two banks (see [14] for details of the game). We simulate 192 trajectories with length 100, with the policies obtained from [14], starting from the 64 different initial states with 3 different initial beliefs of the cop. We set the prior probability distribution as a stationary uniform distribution in the state space.…”

Deceptive Kernel Function on Observations of Discrete POMDP

“…Out of the box of cybersecurity, relative researches across deception and stochastic decision-making process are just emerging in optimal control and reinforcement learning areas. Ornik and Topcu in [11] have introduced a mathematical framework for deception in optimal control based on Markov decision process (MDP). However, the framework has a prerequisite of its deception objective being reward-based.…”

“…While these researches all demonstrate novel and inspirational work from their respective angle, they are meantime subject to several common limitations. Attacks modeled in researches [11], [23] and [14] are subject to certain "objectives" of deception/attack; research in [21] considers human cognitive bias as a reason for agent's potential suboptimal choices; models in [21], [16], [23] and [14] are computed with underlying MDP regardless of agent's partial observability as a potential setting; and some of the tactics introduced above are more attacking rather than deceptive.…”