Exploitation of an opponent's imperfect information in a stochastic game with autonomous vehicle application

Abstract

We consider a finite state space, discrete stochastic game problem where only one player has perfect information. In the notation employed here only the "Red" player has the perfect state information. The "Blue" player only has access to observation-based information. To some degree the observations may be influenced by the controls of both players. A Markov chain model is used where the transition probabilities depend on the controls of the players. The game is zero-sum. It is known that application of the optimal control at a maximum likelihood estimate by the Blue player is not optimal; under a saddle point condition, a form of Certainty Equivalence does exist for the Blue player, but the structure is more complex than the above approach. In this work, the point of view of the Red player is considered. Simulation is used to demonstrate that the optimal state feedback control for Red is not the optimal control (even with perfect information for Red). This is a significantly stronger statement than that Certainty Equivalence does not hold when the Red player has imperfect information. A theory for the development of Red control is presented. This yields "deceptive" controls in the presence of the simpler Blue approach above, which provide superior performance in this case. An open question is whether (and under what conditions), this approach yields superior performance for Red as compared with state feedback when Blue is allowed strategies including the more complex one above. Experimentation and theory will be employed to answer this question.