Decentralized Online Learning for Noncooperative Games in Dynamic Environments

Meng, Meng; Li, Xiuxian; Hong, Youwei; Chen, Jie; Wang, Long

doi:10.48550/arxiv.2105.06200

Cited by 4 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that seeking all GNEs is rather difficult even for offline game, and thereby this paper focuses on seeking the unique variational GNE sequence. As a matter of fact, seeking the variational GNE of games with coupled constraints was widely studied [12], [13], [20] since the variational GNE has good stability with the economic interpretation of no price discrimination. Some assumptions on players' communication are listed below.…”

Section: A Problem Formulationmentioning

confidence: 99%

“…Remark 1: To date, a projection-based distributed algorithm was devised in [12] for online game with time-invariant constraint functions, and then a mirror descent-based distributed algorithm was proposed in [13] for online game with timevarying constraint functions. However, both works depend on that each player can receive the gradients of its local cost and constraint functions after a strategy profile is determined at each round.…”

Section: Distributed Bandit Feedbackmentioning

confidence: 99%

“…In this setting, a distributed GNE learning algorithm for online game with time-invariant constraints was designed in [12] based on primal-dual strategies and distributed consensus. Subsequently, the authors of [13] considered time-varying constraints and proposed a distributed online algorithm on the basis of mirror descent and primaldual strategies.…”

mentioning

confidence: 99%

“…In [12], [13], each player can access the gradient information of its cost and local constraint functions sequentially. However, the gradient information in many realistic applications cannot be grabbed by local players, especially if the cost and constraint functions are not revealed.…”

mentioning

confidence: 99%

“…1) To the best of our knowledge, this paper is the first to study online game with time-varying coupled inequality constraints with bandit feedback, while existing works on online game only consider the case with full information of cost and constraint functions [12], [13]. 2) A distributed GNE seeking algorithm for online game is devised by mirror descent and one-point bandit feedback.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Decentralized Nash Equilibria Learning for Online Game with Bandit Feedback

Mao¹,

Li²,

Chen³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper studies distributed online bandit learning of generalized Nash equilibria for online game, where cost functions of all players and coupled constraints are time-varying. The values rather than full information of cost and local constraint functions are revealed to local players gradually. The goal of each player is to selfishly minimize its own cost function with no future information subject to a strategy set constraint and time-varying coupled inequality constraints. To this end, a distributed online algorithm based on mirror descent and one-point bandit feedback is designed for seeking generalized Nash equilibria of the online game. It is shown that the devised online algorithm achieves sublinear expected regrets and accumulated constraint violation if the path variation of the generalized Nash equilibrium sequence is sublinear. Furthermore, the proposed algorithm is extended to the scenario of delayed bandit feedback, that is, the values of cost and constraint functions are disclosed to local players with time delays. It is also demonstrated that the online algorithm with delayed bandit feedback still has sublinear expected regrets and accumulated constraint violation under some conditions on the path variation and delay. Simulations are presented to illustrate the efficiency of theoretical results.

show abstract

Section: A Problem Formulationmentioning

confidence: 99%

Section: Distributed Bandit Feedbackmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Decentralized Nash Equilibria Learning for Online Game with Bandit Feedback

Mao¹,

Li²,

Chen³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Distributed Nash equilibrium tracking via the alternating direction method of multipliers

Ma,

Yang,

Chen

2023

Kybernetika

View full text Add to dashboard Cite

Nash equilibrium is recognized as an important solution concept in non-cooperative game theory due to its broad applicability to economics, social sciences, computer science, and engineering. In view of its importance, substantial progress has been made to seek a static Nash equilibrium using distributed methods. However, these approaches are inapplicable in dynamic environments because, in this setting, the Nash equilibrium constantly changes over time. In this paper, we propose a dynamic algorithm that can track the time-varying Nash equilibrium in a non-cooperative game. Our approach enables each player to update its action using an alternating direction method of multipliers while ensuring this estimated action of each player always converges to a neighborhood of the Nash equilibrium at each sampling instant. We prove that the final tracking error is linearly proportional to the sampling interval, which implies that the tracking error can be sufficiently close to zero when the sampling interval is small enough. Finally, numerical simulations are conducted to verify the correctness of our theoretical results.

show abstract

A Survey on Distributed Online Optimization and Game

Li¹,

Xie²,

Li³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Decentralized online learning (DOL) has been increasingly researched in the last decade, mostly motivated by its wide applications in sensor networks, commercial buildings, robotics (e.g., decentralized target tracking and formation control), smart grids, deep learning, and so forth. In this problem, there are a network of agents who may be cooperative (i.e., decentralized online optimization) or noncooperative (i.e., online game) through local information exchanges, and the local cost function of each agent is often time-varying in dynamic and even adversarial environments. At each time, a decision must be made by each agent based on historical information at hand without knowing future information on cost functions. Although this problem has been extensively studied in the last decade, a comprehensive survey is lacking. Therefore, this paper provides a thorough overview of DOL from the perspective of problem settings, communication, computation, and performances. In addition, some potential future directions are also discussed in details.

show abstract

Decentralized Online Learning for Noncooperative Games in Dynamic Environments

Cited by 4 publications

References 31 publications

Decentralized Nash Equilibria Learning for Online Game with Bandit Feedback

Decentralized Nash Equilibria Learning for Online Game with Bandit Feedback

Distributed Nash equilibrium tracking via the alternating direction method of multipliers

A Survey on Distributed Online Optimization and Game

Contact Info

Product

Resources

About