Decentralized Control of Two Agents with Nested Accessible Information

Dave, Aditya; Nishanth, Venkatesh,; Malikopoulos, Andreas A.

doi:10.23919/acc53348.2022.9867605

Cited by 6 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related Works 1) Team optimal control & Information structures: Team decision problems, which were first investigated in [5], [6], involve multiple decision makers (DMs) each of whom has access to different information variables and consequently choose policies jointly to incur a common cost/reward. Since each one acts independently, and they do not necessarily share the same information, the joint optimal policy design is highly dependent on the information available to each DM [7]- [9], and its derivation can be challenging particularly when the information is dynamic, as demonstrated by Witsenhausen [10], Feldbaum [11], and Bas ¸ar [12]. The single team communication-control trade-off problem is an instance of such a decision problem, where the sensor and the controller form a 2-DM team, each having access to different information signals within the system.…”

Section: Introductionmentioning

confidence: 99%

Large Population Games with Timely Scheduling over Constrained Networks

Aggarwal

Zaman

Bastopcu

et al. 2023

2023 American Control Conference (ACC)

View full text Add to dashboard Cite

Coordinating communication and control is a key component in the stability and performance of networked multi-agent systems. While single user networked control systems have gained a lot of attention within this domain, in this work, we address the more challenging problem of large population multi-team dynamic games. In particular, each team constitutes two decision makers (namely, the sensor and the controller) who coordinate over a shared network to control a dynamically evolving state of interest under costs on both actuation and sensing/communication. Due to the shared nature of the wireless channel, the overall cost of each team depends on other teams' policies, thereby leading to a noncooperative game setup. Due to the presence of a large number of teams, we compute approximate decentralized Nash equilibrium policies for each team using the paradigm of (extended) mean-field games, which is governed by (1) the mean traffic flowing over the channel, and (2) the value of information at the sensor, which highlights the semantic nature of the ensuing communication. In the process, we compute optimal controller policies and approximately optimal sensor policies for each representative team of the mean-field system to alleviate the problem of general non-contractivity of the mean-field fixed point operator associated with the finite cardinality of the sensor action space.Consequently, we also prove the ǫ-Nash property of the mean-field equilibrium solution which essentially characterizes how well the solution derived using mean-field analysis performs on the finite-team system.Finally, we provide extensive numerical simulations, which corroborate the theoretical findings and lead to additional insights on the properties of the results presented.

show abstract

Section: Introductionmentioning

confidence: 99%

Large Population Games with Timely Scheduling over Constrained Networks

Aggarwal

Zaman

Bastopcu

et al. 2023

2023 American Control Conference (ACC)

View full text Add to dashboard Cite

show abstract

“…Cyber-physical systems, such as connected and automated vehicles [1] and power systems [2], often require decisionmaking in uncertain environments with partial knowledge of the dynamics [3] over long time horizons. This decisionmaking challenge is typically modeled with a stochastic formulation, where an agent can access a prior probability distribution for all uncertainties and computes a control strategy to minimize the expected value of a discounted total cost across an infinite time horizon [4]. However, the actual performance of such a strategy degrades when the given prior distribution is different from the actual underlying distribution [5].…”

Section: Introductionmentioning

confidence: 99%

Approximate Information States for Worst-case Control of Uncertain Systems

Dave

Nishanth

Malikopoulos

2022

2022 IEEE 61st Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

Safety-critical cyber-physical systems require control strategies whose worst-case performance is robust against adversarial disturbances and modeling uncertainties. In this paper, we present a framework for approximate control and learning in partially observed systems to minimize the worstcase discounted cost over an infinite time horizon. We model disturbances to the system as finite-valued uncertain variables with unknown probability distributions. For problems with known system dynamics, we construct a dynamic programming (DP) decomposition to compute the optimal control strategy. Our first contribution is to define information states that improve the computational tractability of this DP without loss of optimality. Then, we describe a simplification for a class of problems where the incurred cost is observable at each time instance. Our second contribution is defining an approximate information state that can be constructed or learned directly from observed data for problems with observable costs. We derive bounds on the performance loss of the resulting approximate control strategy and illustrate the effectiveness of our approach in partially observed decision-making problems with a numerical example.

show abstract

“…, T , as illustrated in Fig.4. At each t, the encoder ψ t comprises of 3 layer neural network with sizes (2, 14),(14,12), (12+24, 24) and ReLU activation for the first two layers, where the inputs are a 2-d vector of coordinates for observation Y t and a 24-d vector for the previous approximate information state Πt−1 . The encoder compresses these inputs to a 24-d vector representing the approximate information state Πt .…”

mentioning

confidence: 99%

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

Dave¹,

Nishanth²,

Malikopoulos³

2023

Preprint

View full text Add to dashboard Cite

In this paper, we investigate discrete-time decisionmaking problems in uncertain systems with partially observed states. We consider a non-stochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by developing the notions of information states and approximate information states. In our definition of an information state, we introduce conditions to identify for an uncertain variable sufficient to construct a dynamic program (DP) that computes an optimal strategy. We show that many information states from the literature on worstcase control actions, e.g., the conditional range, are examples of our more general definition. Next, we relax these conditions to define approximate information states using only output variables, which can be learned from output data without knowledge of system dynamics. We use this notion to formulate an approximate DP that yields a strategy with a bounded performance loss. Finally, we illustrate the application of our results in control and reinforcement learning using numerical examples.

show abstract

Decentralized Control of Two Agents with Nested Accessible Information

Cited by 6 publications

References 18 publications

Large Population Games with Timely Scheduling over Constrained Networks

Large Population Games with Timely Scheduling over Constrained Networks

Approximate Information States for Worst-case Control of Uncertain Systems

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

Contact Info

Product

Resources

About