The Hanabi challenge: A new frontier for AI research

Abstract

From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques. 6 One such equilibrium occurs when players do not intentionally communicate information to other players, and ignore what other players tell them (historically called a pooling equilibrium in pure signalling games [15], or a babbling equilibrium in later work using cheap talk [16]). In this case, there is no incentive for a player to start communicating because they will be ignored, and there is no incentive to pay attention to other players because they are not communicating.7 In pure signalling games where actions are purely communicative, policies are often referred to as communication protocols. Though Hanabi is not such a pure signalling game, when we want to emphasize the communication properties of an agent's policy we will still refer to its communication protocol. 8 We use the word convention to refer to the parts of a communication protocol or policy that interrelate. Technically, these can be thought of as constraints on the policy to enact the convention.

From Chess and Atari to StarCraft and Beyond: How Game AI is Driving the World of AI

“…The most active institutes in the area can be taken from a list (incomplete, focused only on the most relevant venues) compiled by Mark Nelson. 6 A large part of the progress of the last years is due to the free availability of competition environments as: StarCraft, GVGAI, Angry Birds, Hearthstone, Hanabi, MicroRTS, Fighting Game, Geometry Friends and more, and also the more general frameworks as: ALE, GGP, OpenSpiel, OpenAIGym, SC2LE, MuJoCo, Deep-RTS. 6 http://www.kmjn.org/game-rankings 10 The future of Game AI More advanced AI techniques are slowly finding their way into the game industry and this will likely increase significantly over the coming years.…”

“…Recent attempts in this direction include e.g. : Open AI five [64], Hanabi [6], capture the flag [31] -More natural language processing enables better interfaces and at some point free-form direct communication with game characters. Already existing commercial voice-driven assistance systems as the Google Assistant or Alexa show that this is possible.…”

Diverse Agents for Ad-Hoc Cooperation in Hanabi

“…Other than the CoG competition and its related agents, another work on ad-hoc teamplay using Hanabi is by Bard et al [18], who independently trained reinforcement learning agents that scored 20 to 22 points in self-play, but only 0 to 5 when paired with one another. They also proposed an adhoc setting where self-play playtraces of the partner agent are provided prior to gameplay for learning, but no agent currently takes advantage of this feature.…”

“…A pool where fairly simple heuristic agents are mixed with intentionally bad agents as seen in [13] might encourage challengers to identify the bad agents and guard against their bad behaviors, while avoiding strategies that require many assumptions when paired with the regular heuristic agents. On the other hand, a pool of very high-performing selfplay agents with sophisticated conventions (such as the ones developed in [20], [21] or learned by Reinforcement Learning in [8], [18]) might resemble more a puzzle-solving challenge where the exact convention in use must be identified to achieve good performance.…”

“…For example, two agents that score higher when playing with each other than either agent does on self-play might have found a pair of synergistic strategies. Agents in pool with high self-play performances but low pairwise performances could be using strategies that are very effective if shared by both players, but are not suited for ad-hoc play, as is the case in [18].…”