Effective short-term opponent exploitation in simplified poker

Abstract

Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic game-theoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some model or counterstrategy can be learned by observing their play. However, games against humans last for at most a few hundred hands, so learning must be very fast to be useful. We explore two approaches to opponent modelling in the context of Kuhn poker, a small game for which game-theoretic solutions are known. Parameter estimation and expert algorithms are both studied. Experiments demonstrate that, even in this small game, convergence to maximally exploitive solutions in a small number of hands is impractical, but that good (e.g., better than Nash) performance can be achieved in as few as 50 hands. Finally, we show that amongst a set of strategies with equal game-theoretic value, in particular the set of Nash equilibrium strategies, some are preferable because they speed learning of the opponent's strategy by exploring it more effectively.

Multiagent learning in the presence of memory-bounded agents

“…Most notable amongst them are the works on opponent modeling in Poker, an extensive form game [37]. There has also been some research addressing opponent modeling in games which have emerged from popular competitions, such as the Lemonade Stand game [42] and the games from the Trading Agent competitions [31].…”

Simplified three player Kuhn poker

“…Recently, two player Limit Holdem, a large and complex game, was numerically solved in this sense [5], and a recently-developed, sophisticated AI that combines neural networks with on the fly equilibrium computation plays the even larger game of two player No Limit Holdem close to optimally [7]. However, even a simple two player, zero sum game leads to significant theoretical challenges when studied as a repeated game (for example, two player Kuhn poker, also known as the AKQ game, [9]).…”

Game theory models for communication between agents: a review

“…When both players pass or bet then the player with the highest card will win the pot. Southey et al (2009) have studied Kuhn Poker game. There main concern is opponent modeling in the game.…”