Strategy Synthesis for Zero-sum Neuro-symbolic Concurrent Stochastic Games (Extended Version)

Abstract

Neuro-symbolic approaches to artificial intelligence, which combine neural networks with classical symbolic techniques, are growing in prominence, necessitating formal approaches to reason about their correctness. We propose a novel modelling formalism called neuro-symbolic concurrent stochastic games (NS-CSGs), which comprise a set of probabilistic finite-state agents interacting in a shared continuous-state environment, observed through perception mechanisms implemented as neural networks. Since the environment state space is continuous, we focus on the class of NS-CSGs with Borel state spaces and Borel measurability restrictions on the components of the model.We consider the problem of zero-sum discounted cumulative reward, proving that NS-CSGs are determined and therefore have a value which corresponds to a unique fixed point. From an algorithmic perspective, existing methods to compute values and optimal strategies for CSGs focus on finite state spaces. We present, for the first time, value iteration and policy iteration algorithms to solve a class of uncountable state space CSGs, and prove their convergence. Our approach works by formulating piecewise linear or constant representations of the value functions and strategies of NS-CSGs. We validate the approach with a prototype implementation applied to a dynamic vehicle parking example.

Finite-horizon Equilibria for Neuro-symbolic Concurrent Stochastic Games

“…We begin by describing neuro-symbolic concurrent stochastic games (NS-CSGs) [Yan et al, 2022b], the modelling formalism that we use in this paper, for which we then define our notions of equilibria.…”

“…Formally, the semantics of an NS-CSG is an infinite state CSG, as described in [Yan et al, 2022b]. For brevity, we present here an informal description of the semantics.…”

“…In particular, concurrent stochastic games (CSGs) provide a natural framework for modelling a set of interactive, rational agents operating concurrently within an uncertain or probabilistic environment. For finitestate CSGs, algorithms for their solution are known [Chatterjee et al, 2013, de Alfaro and Majumdar, 2004, De Alfaro et al, 2007 and, more recently, techniques and tools for In this paper, we work with the recently proposed formalism of neuro-symbolic concurrent stochastic games (NS-CSGs) [Yan et al, 2022b], designed to model probabilistic multi-agent systems comprising neuro-symbolic agents operating concurrently within a shared, continuous-state environment. In [Yan et al, 2022b], the zero-sum control problem is considered, namely to synthesise strategies for one set of agents who are aiming to maximise their (discounted, infinite-horizon) expected reward, while the other agents aim to minimise this value.…”

“…For finitestate CSGs, algorithms for their solution are known [Chatterjee et al, 2013, de Alfaro and Majumdar, 2004, De Alfaro et al, 2007 and, more recently, techniques and tools for In this paper, we work with the recently proposed formalism of neuro-symbolic concurrent stochastic games (NS-CSGs) [Yan et al, 2022b], designed to model probabilistic multi-agent systems comprising neuro-symbolic agents operating concurrently within a shared, continuous-state environment. In [Yan et al, 2022b], the zero-sum control problem is considered, namely to synthesise strategies for one set of agents who are aiming to maximise their (discounted, infinite-horizon) expected reward, while the other agents aim to minimise this value. However, in practice, this is limiting: even for the case of just two coalitions of agents, they will often have distinct, but not directly opposing goals, which cannot be modelled in a zero-sum fashion.…”

“…So we then propose frozen subgame improvement (FSI), an approximation algorithm which iteratively solves nonlinear programs to monotonically improve the social welfare. Our approach is wholly different from the zero-sum (discounted, infinite-horizon) solution of NS-CSGs in [Yan et al, 2022b], which applies value/policy iteration to finite model abstractions that rely on assumptions about the functions used to specify the model.…”