Double-Oracle Sampling Method for Stackelberg Equilibrium Approximation in General-Sum Extensive-Form Games

Abstract

The paper presents a new method for approximating Strong Stackelberg Equilibrium in general-sum sequential games with imperfect information and perfect recall. The proposed approach is generic as it does not rely on any specific properties of a particular game model. The method is based on iterative interleaving of the two following phases: (1) guided Monte Carlo Tree Search sampling of the Follower's strategy space and (2) building the Leader's behavior strategy tree for which the sampled Follower's strategy is an optimal response. The above solution scheme is evaluated with respect to expected Leader's utility and time requirements on three sets of interception games with variable characteristics, played on graphs. A comparison with three state-of-the-art MILP/LP-based methods shows that in vast majority of test cases proposed simulation-based approach leads to optimal Leader's strategies, while excelling the competitive methods in terms of better time scalability and lower memory requirements.

Anchoring Theory in Sequential Stackelberg Games

“…The first approach (referred to as O2UCT -double-oracle UCT sampling) [11,13] relies on a guided sampling of the follower's strategy space interleaved with finding a feasible leader's strategy using double-oracle method.…”

“…δ l must satisfy the following conditions: (1) π r f is the best response strategy against δ l ; (2) δ l provides as high as possible leader's utility when played against the best follower's response. An algorithm of finding the requested leader's strategy δ l is outlined below and detailed in [13].…”

“…At the same time, in reference to large-scale sequential SGs, several algorithms utilizing different techniques, e.g. sequence-form [2], correlated equilibria [4], game abstraction [5], Evolutionary Algorithm [12,34] or Monte Carlo sampling [10,13] which visibly extended the range of tractable SGs, have been proposed recently.…”

“…Consequently, modifications to three state-of-the-art methods for solving extensive-form SSGs [4,5,2] which implement AT are proposed. Furthermore, two other non-MILP heuristic methods for solving SSG that rely on Monte Carlo sampling [11,13] and Evolutionary Algorithm [34], respectively are also adequately modified to incorporate AT principles. All five methods are experimentally evaluated on a set of Warehouse Games [10,16].…”

Learning Attacker's Bounded Rationality Model in Security Games

“…CBK2018 [17]), others employ different ideas. O2UCT [18,19], For instance, utilizes Upper Confidence Bounds applied to trees [20] (a variant of Monte Carlo Tree Search) and combines sampling the follower's strategy space with calculating the best leader's strategy for which a sampled followers strategy is the optimal response. Another heuristic method, EASG [21] maintains a population of candidate leader's strategies and applies specifically designed mutation and crossover operators.…”