We consider a class of regime-switching noncooperative repeated games where agents exchange information over a graph. The parameters of the game (number of agents, payoffs, information exchange graph) evolve randomly over time according to a Markov chain. We present a regret-based stochastic approximation algorithm with constant step-size that prescribes how individual agents update their randomized strategies over time. We show that, if the Markov chain jump changes on the same timescale as the adaptation rate of the stochastic approximation algorithm and agents independently follow this algorithm, their collective behavior is agile in tracking the time-varying convex polytope of correlated equilibria. The analysis is carried out using weak convergence methods and Lyapunov stability of switched Markovian differential inclusions.Index Terms-Coordination in decision making, correlated equilibrium, Lyapunov stability, Markov chain, noncooperative game, regime-switching differential inclusion, stochastic approximation, tracking.
0018-9286
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.