A Monte Carlo Tree Search approach to finding efficient patrolling schemes on graphs

Karwowski, Jan; Mańdziuk, Jacek

doi:10.1016/j.ejor.2019.02.017

Cited by 19 publications

(20 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We refer to problems that have this structure as Stackelberg security games (SSGs), which are introduced in detail in Section 2. Some SSG applications have included assigning Federal Air Marshals to transatlantic flights [Jain et al, 2010], determining randomized port and waterways patrols for the U.S. Coast Guard [Shieh et al, 2012], preventing fare evasion in public transport systems , protecting endangered wildlife [Yang et al, 2014], and patrolling applications to protect networked infrastructure [Karwowski andMańdziuk, 2019, Li et al, 2019]. The SSG models considered are closely related to the interdiction games literature, [McMasters and Mustin, 1970], specifically when there is a fortification step.…”

Section: Introductionmentioning

confidence: 99%

A study of general and security Stackelberg game formulations

Casorrán

Fortz

Ordóñez

2019

European Journal of Operational Research

View full text Add to dashboard Cite

In this paper, we analyze different mathematical formulations for general Stackelberg games (GSGs) and Stackelberg security games (SSGs). We consider GSGs in which a single leader commits to a utility maximizing strategy knowing that p possible followers optimize their own utility taking the leader's strategy into account. SSGs are a type of GSG that arise in security applications where the strategies of the leader consist of protecting a subset of targets and the strategies of the p followers consist of attacking a single target. We compare existing mixed integer linear programming (MILP) formulations for GSGs, ranking them according to the tightness of their linear programming (LP) relaxations. We show that SSG formulations are projections of GSG formulations and exploit this link to derive a new SSG MILP formulation that i) has the tightest LP relaxation known among SSG MILP formulations and ii) has an LP relaxation that coincides with the convex hull of feasible solutions in the case of a single follower. We present computational experiments empirically comparing the difficulty of solving the formulations in the general and security settings. The new SSG MILP formulation remains computationally efficient as problem size increases.

show abstract

Section: Introductionmentioning

confidence: 99%

A study of general and security Stackelberg game formulations

Casorrán

Fortz

Ordóñez

2019

European Journal of Operational Research

View full text Add to dashboard Cite

show abstract

“…Bertsimas et al (2017) apply MCTS to two large-scale dynamic resource allocation problems dealing with tactical wildfire management and queuing network control. An MCTS approach to find efficient patrolling schemes on graphs is proposed by Karwowski and Mańdziuk (2019). Neto et al (2020) use MCTS for a harvest scheduling problem.…”

Section: Monte Carlo Tree Searchmentioning

confidence: 99%

Artificial intelligence-based inventory management: a Monte Carlo tree search approach

Preil

Krapp

2021

Ann Oper Res

View full text Add to dashboard Cite

The coordination of order policies constitutes a great challenge in supply chain inventory management as various stochastic factors increase its complexity. Therefore, analytical approaches to determine a policy that minimises overall inventory costs are only suitable to a limited extent. In contrast, we adopt a heuristic approach, from the domain of artificial intelligence (AI), namely, Monte Carlo tree search (MCTS). To the best of our knowledge, MCTS has neither been applied to supply chain inventory management before nor is it yet widely disseminated in other branches of operations research. We develop an offline model as well as an online model which bases decisions on real-time data. For demonstration purposes, we consider a supply chain structure similar to the classical beer game with four actors and both stochastic demand and lead times. We demonstrate that both the offline and the online MCTS models perform better than other previously adopted AI-based approaches. Furthermore, we provide evidence that a dynamic order policy determined by MCTS eliminates the bullwhip effect.

show abstract

“…To the best of our knowledge there is only one other approach that utilizes MCTS method to solve general-sum extensive-form SGs (Karwowski and Mańdziuk 2015;2019a) which, however, adopts a different protocol and relies on iterative adjustment of the Leader's strategy by means of direct MCTS sampling against gradually changing Follower's strategy. This method, though, could not be applied to solve games with complex information set (IS) structures, e.g.…”

Section: Contributionmentioning

confidence: 99%

“…Evaluation of O2UCT was performed on three game sets: Warehouse Games proposed in (Karwowski and Mańdziuk 2019a), its modified version with more diverse payoffs, and Search Games used in (Bosansky and Cermak 2015).…”

Section: Experimental Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Double-Oracle Sampling Method for Stackelberg Equilibrium Approximation in General-Sum Extensive-Form Games

Karwowski

Mańdziuk

2020

AAAI

Self Cite

View full text Add to dashboard Cite

The paper presents a new method for approximating Strong Stackelberg Equilibrium in general-sum sequential games with imperfect information and perfect recall. The proposed approach is generic as it does not rely on any specific properties of a particular game model. The method is based on iterative interleaving of the two following phases: (1) guided Monte Carlo Tree Search sampling of the Follower's strategy space and (2) building the Leader's behavior strategy tree for which the sampled Follower's strategy is an optimal response. The above solution scheme is evaluated with respect to expected Leader's utility and time requirements on three sets of interception games with variable characteristics, played on graphs. A comparison with three state-of-the-art MILP/LP-based methods shows that in vast majority of test cases proposed simulation-based approach leads to optimal Leader's strategies, while excelling the competitive methods in terms of better time scalability and lower memory requirements.

show abstract

A Monte Carlo Tree Search approach to finding efficient patrolling schemes on graphs

Cited by 19 publications

References 35 publications

A study of general and security Stackelberg game formulations

A study of general and security Stackelberg game formulations

Artificial intelligence-based inventory management: a Monte Carlo tree search approach

Double-Oracle Sampling Method for Stackelberg Equilibrium Approximation in General-Sum Extensive-Form Games

Contact Info

Product

Resources

About