Optimal Policies for Convex Symmetric Stochastic Dynamic Teams and their Mean-Field Limit

Sanjari, Sina; Yüksel, Serdar

doi:10.1137/19m1284294

Cited by 11 publications

(9 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that if one also has convexity in the cost as well as action sets U i , then one can also establish that for every finite N , the optimal policies are symmetric and deterministic, but in the infinite limit, randomization may be required [122]. Similar results also hold for dynamic team problems [123]. We emphasize that a strategic measures approach would not be feasible for arriving at this solution since exchangeability in the actions is not sufficient to ensure that the dominating random variable (in the de Finetti representation) is independent of the intrinsic randomness in the system.…”

Section: = Infmentioning

confidence: 60%

Geometry of information structures, strategic measures and associated stochastic control topologies

Saldi

Yüksel

2022

Probab. Surveys

Self Cite

View full text Add to dashboard Cite

In many areas of applied mathematics, decentralization of information is a ubiquitous attribute affecting how to approach a stochastic optimization, decision and estimation, or control problem. In this review article, we present a general formulation of information structures under a probability theoretic and geometric formulation. We define information structures, place various topologies on them, and study closedness, compactness and convexity properties on the strategic measures induced by information structures and decentralized control/decision policies under varying degrees of relaxations with regard to access to private or common randomness. Ultimately, we present existence and tight approximation results for optimal decision/control policies. We discuss various lower bounding techniques, through relaxations and convex programs ranging from classically realizable and classically non-realizable (such as quantum theoretic and non-signaling) relaxations. For each of these, we establish closedness and convexity properties and also a hierarchy of correlation structures. As a further theme, we review and introduce various topologies on decision/control strategies defined independent of information structures, but for which information structures determine whether the topologies entail utility in arriving at existence, compactness, convexification or approximation results. These approaches, which we term as the strategic measures approach and the control topology approach, lead to complementary results on existence, approximations and upper and lower bounds in optimal decentralized stochastic decision, estimation, and control problems.

show abstract

Section: = Infmentioning

confidence: 60%

Geometry of information structures, strategic measures and associated stochastic control topologies

Saldi

Yüksel

2022

Probab. Surveys

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the following this leads to the definition of a player, which is a collection of DMs acting through time. With this motivation, we will define a new static reduction concept building on the one introduced by Witsenhausen (called independent-data reduction) [35,Section 2.4] and the other one in [29,Section 3.2]. The underlying idea is to view agents acting in sequence with increasing information as a single player with a larger action space.…”

Section: E Lqg Team Problems With a Partially Nested Ismentioning

confidence: 99%

“…This facilitates our optimality analysis. We note that this approach was utilized to establish structural and existence results in [29,Section 3.2]. Now, we define multi-stage team problems and optimality concepts for these problems.…”

Section: E Lqg Team Problems With a Partially Nested Ismentioning

confidence: 99%

“…Remark 9. Mean-field teams can be viewed as limit models of symmetric finite player teams with a mean-field interaction (for example, see [29], [28] for mean-field teams, and [9] and references therein for mean-field games). We note that for multi-stage mean-field dynamic teams, the independent data and nested static reduction have been introduced in [29, Section 3.2] and [28,Assumption 5.1(ii)].…”

mentioning

confidence: 99%

“…We note that for multi-stage mean-field dynamic teams, the independent data and nested static reduction have been introduced in [29, Section 3.2] and [28,Assumption 5.1(ii)]. As it has been shown in [29,Section 3.2] and [28,Assumption 5.1(ii)], the above static reduction under mild assumptions on the action and observation spaces leads to a closedness of a set of policies for each player through times under an appropriate topology, which is desirable for establishing existence and/or convergence results. We also note that the infinite horizon team problem under a nested static reduction is more tractable compared to an independent-data static reduction.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Isomorphism Properties of Optimality and Equilibrium Solutions under Equivalent Information Structure Transformations I: Stochastic Dynamic Teams

Sanjari¹,

Başar²,

Yüksel³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Static reduction of dynamic stochastic team (or decentralized stochastic control) problems has been an effective method for establishing existence and approximation results for optimal policies. In this Part I of a two-part paper, we address stochastic dynamic teams. Part II addresses stochastic dynamic games. We consider two distinct types of static reductions: (i) those that are policy-independent (as those introduced by Witsenhausen in [35]), and (ii) those that are policy-dependent (as those introduced by Ho and Chu [15], [16] for partially nested dynamic teams). For the first type, we show that there is a bijection between stationary (person-by-person optimal, globally optimal) policies and their policyindependent static reductions, and then we present a variational analysis for convex dynamic teams under policy-independent static reductions. For the second type, although there is a bijection between globally optimal policies of dynamic teams with partially nested information structures and their static reductions, in general there is no bijection between stationary (person-by-person optimal) policies of dynamic teams and their policy-dependent static reductions. We present, however, sufficient conditions under which such a relationship holds and a functional form of the bijection can be constructed. Furthermore, we study the connections between optimality and stationarity concepts when a control-sharing information structure is considered (via an expansion of the information structures) under policy-dependent static reductions. An implication is a convexity characterization of dynamic team problems under policy-dependent static reduction with a control-sharing information structure. Finally, we study multi-stage team problems, where we discuss the connections between optimality concepts through times and players under the introduced static reductions.

show abstract

Information Structures in Decentralized Stochastic Control

Yüksel,

Başar

2024

Systems &Amp; Control: Foundations &Amp; Applications

View full text Add to dashboard Cite

Optimal Policies for Convex Symmetric Stochastic Dynamic Teams and their Mean-Field Limit

Cited by 11 publications

References 31 publications

Geometry of information structures, strategic measures and associated stochastic control topologies

Geometry of information structures, strategic measures and associated stochastic control topologies

Isomorphism Properties of Optimality and Equilibrium Solutions under Equivalent Information Structure Transformations I: Stochastic Dynamic Teams

Information Structures in Decentralized Stochastic Control

Contact Info

Product

Resources

About