COG-DICE: An Algorithm for Solving Continuous-Observation Dec-POMDPs

Abstract

The decentralized partially observable Markov decision process (Dec-POMDP) is a powerful model for representing multi-agent problems with decentralized behavior. Unfortunately, current Dec-POMDP solution methods cannot solve problems with continuous observations, which are common in many real-world domains. To that end, we present a framework for representing and generating Dec-POMDP policies that explicitly include continuous observations. We apply our algorithm to a novel tagging problem and an extended version of a common benchmark, where it generates policies that meet or exceed the values of equivalent discretized domains without the need for finding an adequate discretization.

Privacy-Preserving Policy Iteration for Decentralized POMDPs

“…Specifically, our solution is model-free where each agent's policy is optimized iteratively with the local information collected in several trials by that agent. In more detail, we optimize each agent's policy using a variation of the Cross-Entropy (CE) method (Oliehoek, Kooij, and Vlassis 2008;Omidshafiei et al 2016;Clark-Turner and Amato 2017). Like most of the existing algorithms for Dec-POMDPs, privacy issues are not concerned in the vanilla CE method (Oliehoek, Kooij, and Vlassis 2008).…”