Heads-up limit hold’em poker is solved

Abstract

Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events. Whereas many perfect information games have been solved (e.g., Connect Four and checkers), no nontrivial imperfect information game played competitively by humans has previously been solved. Here, we announce that heads-up limit Texas hold'em is now essentially weakly solved. Furthermore, this computation formally proves the common wisdom that the dealer in the game holds a substantial advantage. This result was enabled by a new algorithm, CFR + , which is capable of solving extensive-form games orders of magnitude larger than previously possible.Games have been intertwined with the earliest developments in computation, game theory, and artificial intelligence (AI). At the very conception of computing, Babbage had detailed plans for an "automaton" capable of playing tic-tac-toe and dreamt of his Analytical Engine playing chess (1). Both Turing (2) and Shannon (3) -on paper and in hardware, respectively -developed programs to play chess as a means of validating early ideas in computation and AI. For more than a half century, games have continued to act as testbeds for new ideas and the resulting successes have marked important milestones in the progress of AI. Examples include the checkers-playing computer program Chinook becoming the first to win a world championship title against humans (4), Deep Blue defeating Kasparov in chess (5), and Watson defeating Jennings and Rutter on Jeopardy! (6). However, defeating top human players is not the same as "solving" a game -that is, computing a game-theoretically optimal solution that is incapable of losing against any opponent in a fair game. Notable milestones in the advancement of AI have been achieved through solving games such as Connect Four (7) and checkers (8).Every nontrivial game played competitively by humans that has been solved to date is a perfect-information game (9). In perfect-information games, all players are informed of everything that has occurred in the game before making a decision. Chess, checkers, and backgammon are examples of perfect-information games. In imperfect-information games, players do 1

Neural network optimtzation of algorithm DeepStack for playing in Leduc Hold’em

“…9. Для порівняння вико-ристовуються алгоритм з роботи [5] (умовне позна-чення на гістограмі Program 1), програма з роботи [11] (умовне позначення Program 2) та система з роботи [3] (умовне позначення Program 3).…”

“…Різниця між найкращим для даного порівняння результатом та вразливістю стра-тегії DeepStack складає приблизно 30%. Причиною такої різниці може слугувати те, що автори праці [11] використовували для розрахунку алгоритм CFR+, що є модифікацією CFR, який використовується в DeepStack.…”

Economic reasoning and artificial intelligence

“…A major milestone in computer poker was achieved in 2014 with the effective solution of "heads up limit hold'em" (HULHE), which is a standard two-player version of the most popular poker game (23). HULHE is the largest game of imperfect information ever solved (with more than 10 13 information sets after removing symmetries) and the first imperfect-information game widely played by humans to be solved.…”

Computing Human-Understandable Strategies: Deducing Fundamental Rules of Poker Strategy

“…For example, creators of the optimal strategy for two-player limit Texas hold'em recently wrote, "Overall, we require less than 11 TB [terabytes] of storage to store the regrets and 6 TB to store the average strategy during the computation, which is distributed across a cluster of computation nodes. This amount is infeasible to store in main memory,..." [2]. While such approaches lead to very strong computer agents, it is difficult to see how a human could implement these strategies.…”