Recursive stochastic games with positive rewards

Etessami, Kousha; Wojtczak, Dominik; Yannakakis, Mihalis

doi:10.1016/j.tcs.2018.12.018

Cited by 2 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [7], it was shown that the approximation of the optimal probability of extinction for BMDPs can be done in polynomial time. The computational complexity of computing the optimal expected total cost before extinction for BMDPs follows from [8] and was shown there to be computable in polynomial time via a linear program formulation. The problem of maximising the probability of reaching a state with an entity of a given type for BMDPs was studied in [6].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Hahn

Perez

Schewe

et al. 2021

Computer Aided Verification

Self Cite

View full text Add to dashboard Cite

We study reinforcement learning for the optimal control of Branching Markov Decision Processes (BMDPs), a natural extension of (multitype) Branching Markov Chains (BMCs). The state of a (discrete-time) BMCs is a collection of entities of various types that, while spawning other entities, generate a payoff. In comparison with BMCs, where the evolution of a each entity of the same type follows the same probabilistic pattern, BMDPs allow an external controller to pick from a range of options. This permits us to study the best/worst behaviour of the system. We generalise model-free reinforcement learning techniques to compute an optimal control strategy of an unknown BMDP in the limit. We present results of an implementation that demonstrate the practicality of the approach.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Following [8], we define here a linear equation system with a minimum operator whose Least Fixed Point solution yields the desired optimal values for each type of a BMDP with non-negative costs. This system generalises the Bellman's equations for finite-state MDPs.…”

Section: Fixed Point Equationsmentioning

confidence: 99%

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Hahn

Perez

Schewe

et al. 2021

Computer Aided Verification

Self Cite

View full text Add to dashboard Cite

show abstract

“…Supermodular games always have a pure Nash equilibrium. For related work, see, e.g.,[14,24] or [27, Section 2.1].…”

mentioning

confidence: 99%

The Contest Game for Crowdsourcing Reviews

Mavronicolas¹,

Spirakis²

2023

Preprint

View full text Add to dashboard Cite

We consider a contest game modelling a contest where reviews for m proposals are crowdsourced from n players. Player i has a skill s iℓ for reviewing proposal ℓ; for her review, she strategically chooses a quality q ∈ {1, 2, . . . , Q} and pays an effort fq ≥ 0, strictly increasing with q. For her effort, she is given a payment determined by a payment function, which is either player-invariant, like, e.g., the popular proportional allocation, or player-specific. The cost incurred to player i for each of her reviews is the difference of a skill-effort function Λ(si, fq) minus her payment. Skills may vary for arbitrary players and arbitrary proposals. A proposal-indifferent player i has identical skills: s iℓ = si for all ℓ; anonymous players means si = 1 for all players i. In a pure Nash equilibrium, no player could unilaterally reduce her cost by switching to a different quality. We present three main results:We present a novel potential function to show that the contest game has always a pure Nash equilibrium for the model of arbitrary players and arbitrary proposals with a player-invariant payment function. A particular case of this result answers an intriguing open question from [4]. In contrast, inexistence is possible for a player-specific payment function; the corresponding decision problem is N P-complete. We exploit an increasing-differences property of the skill-effort function to devise, for constant Q, a polynomial-time Θ(n Q ) algorithm for arbitrary players and arbitrary proposals, under a player-invariant payment function, to compute a pure Nash equilibrium; it is a special case of a Θ max{Q 2 , n} n Q − 1 algorithm for arbitrary Q that we present. This settles the parameterized complexity of the problem with respect to the parameter Q. The computed equilibrium is contiguous: players with better skills are contiguously assigned to lower qualities; contiguity is the crux to bypass the exponential barrier incurred when enumerating all profiles. A Θ(max{Q, n}) algorithm for proposal-indifferent and anonymous players, under proportional allocation, for the special case where Λ(si, fq) = si fq and for a concrete scenario of mandatory participation of players in the contest. Starting with the two highest qualities, we greedily proceed to the lowest, focusing each time on a pair of qualities: maintaining players previously assigned to higher qualities, we split the players assigned to the higher of the two between the two qualities currently considered so as to enforce equilibrium.These results are complemented with extentions in various directions; for example, we devise simple Θ(1) algorithms under proportional allocation, taking Λ(si, fq) = si fq and making stronger assumptions on skills and efforts for both arbitrary and proposal-indifferent and anonymous players.

show abstract

Recursive stochastic games with positive rewards

Cited by 2 publications

References 31 publications

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Model-Free Reinforcement Learning for Branching Markov Decision Processes

The Contest Game for Crowdsourcing Reviews

Contact Info

Product

Resources

About