Bounds and dynamics for empirical game theoretic analysis

Abstract

This paper provides several theoretical results for empirical game theory. Specifically, we introduce bounds for empirical game theoretical analysis of complex multi-agent interactions. In doing so we provide insights in the empirical meta game showing that a Nash equilibrium of the estimated meta-game is an approximate Nash equilibrium of the true underlying metagame. We investigate and show how many data samples are required to obtain a close enough approximation of the underlying game. Additionally, we extend the evolutionary dynamics analysis of meta-games using heuristic payoff tables (HPTs) to asymmetric games. The stateof-the-art has only considered evolutionary dynamics of symmetric HPTs in which agents have access to the same strategy sets and the payoff structure is symmetric, implying that agents are interchangeable. Finally, we carry out an empirical illustration of the generalised method in several domains, illustrating the theory and evolutionary dynamics of several versions of the AlphaGo algorithm (symmetric), the dynamics of the Colonel Blotto game played by human players on Facebook (symmetric), the dynamics of several teams of players in the capture the flag game (symmetric), and an example of a meta-game in Leduc Poker (asymmetric), generated by the policy-space response oracle multi-agent learning algorithm. Keywords Empirical games • Asymmetric games • Replicator dynamics 1 Introduction Using game theory to examine multi-agent interactions in complex systems is a non-trivial task, especially when a payoff table or normal form representation is not directly available. Works by Walsh et al. [39,40], Wellman et al. [43,44], and Phelps et al. [23], have shown the great potential of using heuristic strategies and empirical game theory to examine such B Karl Tuyls

α-Rank: Multi-Agent Evaluation by Evolution

“…In this section, we concisely outline the game-theoretic concepts and methods necessary to understand the remainder of the paper. For a detailed discussion of the concepts we refer the reader to [8,13,30,31]. We also introduce a novel game-theoretic concept, Markov-Conley chains, which we use to theoretically ground our results in.…”

“…This evaluation problem has been studied in Empirical Game Theory using the concept of empirical games or meta-games, and the convergence of their dynamics to Nash equilibria [6][7][8][9]. A meta-game is an abstraction of the underlying game, which considers meta-strategies rather than primitive actions [6,8]. In the Go domain, for example, meta-strategies may correspond to different AlphaGo agents (e.g., each meta-strategy is an agent using a set of specific training hyperparameters, policy representations, and so on).…”

“…While much progress has been made in learning for games such as Go [1,2] and Chess [3], computational gains are now enabling algorithmic innovations in domains of significantly higher complexity, such as Poker [4] and MuJoCo soccer [5] where ranking of agents is much more intricate than in classical simple matrix games. With multi-agent learning domains of interest becoming increasingly more complex, we need methods for evaluation and ranking that are both comprehensive and theoretically well-grounded.Evaluation of agents in a multi-agent context is a hard problem due to several complexity factors: strategy and action spaces of players quickly explode (e.g., multi-robot systems), models need to be able to deal with intransitive behaviors (e.g., cyclical best-responses in Rock-Paper-Scissors, but at a much higher scale), the number of agents can be large in the most interesting applications (e.g., Poker), types of interactions between agents may be complex (e.g., MuJoCo soccer), and payoffs for agents may be asymmetric (e.g., a board-game such as Scotland Yard).This evaluation problem has been studied in Empirical Game Theory using the concept of empirical games or meta-games, and the convergence of their dynamics to Nash equilibria [6][7][8][9]. A meta-game is an abstraction of the underlying game, which considers meta-strategies rather than primitive actions [6,8].…”

“…With multi-agent learning domains of interest becoming increasingly more complex, we need methods for evaluation and ranking that are both comprehensive and theoretically well-grounded.Evaluation of agents in a multi-agent context is a hard problem due to several complexity factors: strategy and action spaces of players quickly explode (e.g., multi-robot systems), models need to be able to deal with intransitive behaviors (e.g., cyclical best-responses in Rock-Paper-Scissors, but at a much higher scale), the number of agents can be large in the most interesting applications (e.g., Poker), types of interactions between agents may be complex (e.g., MuJoCo soccer), and payoffs for agents may be asymmetric (e.g., a board-game such as Scotland Yard).This evaluation problem has been studied in Empirical Game Theory using the concept of empirical games or meta-games, and the convergence of their dynamics to Nash equilibria [6][7][8][9]. A meta-game is an abstraction of the underlying game, which considers meta-strategies rather than primitive actions [6,8]. In the Go domain, for example, meta-strategies may correspond to different AlphaGo agents (e.g., each meta-strategy is an agent using a set of specific training hyperparameters, policy representations, and so on).…”

“…Existing meta-game analysis techniques, however, are still limited in a number of ways: either a low number of players or a low number of agents (i.e., meta-strategies) may be analyzed [6,[8][9][10]. Specifically, on the one hand continuous-time meta-game evaluation models, using replicator dynamics from Evolutionary Game Theory [11][12][13][14][15], are deployed to capture the micro-dynamics of interacting agents.…”