Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

Nayyar, Ashutosh; Mahajan, Aditya; Teneketzis, Demosthenis

doi:10.1109/tac.2013.2239000

Cited by 240 publications

(335 citation statements)

References 39 publications

(60 reference statements)

Supporting

Mentioning

327

Contrasting

Order By: Relevance

“…A more recent development for team games exploits when players have some measurement information in common (56,57). To establish the approach, consider a variation of the setup in Section 4.1.…”

Section: Common Informationmentioning

confidence: 99%

See 1 more Smart Citation

Game Theory and Control

Marden

Shamma

2018

Annu. Rev. Control Robot. Auton. Syst.

View full text Add to dashboard Cite

Game theory is the study of decision problems in which there are multiple decision makers and the quality of a decision maker's choice depends on both that choice and the choices of others. While game theory has been studied predominantly as a modeling paradigm in the mathematical social sciences, there is a strong connection to control systems in that a controller can be viewed as a decision-making entity. Accordingly, game theory is relevant in settings with multiple interacting controllers. This article presents an introduction to game theory, followed by a sampling of results in three specific control theory topics where game theory has played a significant role: (a) zero-sum games, in which the two competing players are a controller and an adversarial environment; (b) team games, in which several controllers pursue a common goal but have access to different information; and (c) distributed control, in which both a game and online adaptive rules are designed to enable distributed interacting subsystems to achieve a collective objective.

show abstract

Section: Common Informationmentioning

confidence: 99%

“…The approach of Nayyar et al (56,57) is to transform the team problem into a centralized optimization as follows. First, let z 0 , z 1 , z 2 , u 1 , and u 2 take values in the sets Z 0 , Z 1 , Z 2 , U 1 , and U 2 , respectively.…”

Section: Common Informationmentioning

confidence: 99%

Game Theory and Control

Marden

Shamma

2018

Annu. Rev. Control Robot. Auton. Syst.

View full text Add to dashboard Cite

show abstract

“…Recently, major strides in solving Dec-POMDPs have been made. In particular, it has been shown that there is a reduction from Dec-POMDP to a special type of centralized POMDP called a non-observable Markov decision process (NOMDP) (MacDermed and Isbell, 2013;Nayyar et al, 2013;Dibangoye et al, 2013;Oliehoek and Amato, 2014). This allows POMDP solution methods to be employed in the context of DecPOMDPs.…”

Section: Other Decision Problemsmentioning

confidence: 99%

Multi-objective decision-theoretic planning

Roijers

2016

AI Matters

View full text Add to dashboard Cite

Decision making is hard. It o en requires reasoning about uncertain environments, partial observability and action spaces that are too large to enumerate. In such complex decisionmaking tasks decision-theoretic agents, that can reason about their environments on the basis of mathematical models and produce policies that optimize the utility for their users, can o en assist us.In most research on decision-theoretic agents, the desirability of actions and their e ects is codi ed in a scalar reward function. However, many real-world decision problems have multiple objectives. In such cases the problem is more naturally expressed using a vector-valued reward function. Rather than having a single optimal policy, we then want to produce a set of policies that covers all possible preferences between the objectives. We call such a set a coverage set. In this dissertation, we focus on decision-theoretic planning algorithms that produce the convex coverage set (CCS), which is the optimal solution set when either: 1) the user utility can be expressed as a weighted sum over the values for each objective; or 2) policies can be stochastic.We propose new methods based on two popular approaches to creating planning algorithms that produce an (approximate) CCS by building on an existing single-objective algorithm. In the inner loop approach, we replace the summations and maximizations in the inner most loops of the single-objective algorithm by cross-sums and pruning operations. In the outer loop approach, we solve a multi-objective problem as a series of scalarized problems by employing the single-objective method as a subroutine.Our most important contribution is an outer loop framework that we call optimistic linear support (OLS). As an outer loop method OLS builds the CCS incrementally. We show that, contrary to existing outer loop methods, each intermediate result is a bounded approximation of the CCS with known bounds (even when the single-objective method used is a bounded approximate method as well) and is guaranteed to terminate in a nite number of iterations.We apply OLS-based algorithms to a variety of multi-objective decision problems, and show that it is more memory-e cient, and faster than corresponding inner loop algorithms for moderate numbers of objectives. We show that exchanging subroutines in OLS is relatively easy and illustrate the importance on a complex planning problem. Finally, we show that it is o en possible to reuse parts of the policies and values, found in earlier iterations of OLS, to hot-start later iterations of OLS. Using this last insight, we propose the rst method for multi-objective POMDPs that employs point-based planning and can produce an ε-CCS in reasonable time.Overall, the methods we propose bring us closer to truly practical multi-objective decisiontheoretic planning.

show abstract

“…Therefore, linear control strategies are not necessarily optimal for Problem 1. Our approach to Problem 1 is based on the common information approach [29] for decentralized decision-making. We identify the common belief of the system state for C L and C R .…”

Section: System Model and Problem Formulationmentioning

confidence: 99%

“…The results of [29] are developed only for finite spaces. Therefore, we can not directly apply the results of [29] to Problem 1.…”

Section: System Model and Problem Formulationmentioning

confidence: 99%

Optimal local and remote controllers with unreliable communication

Ouyang

Asghari

Nayyar

2016

2016 IEEE 55th Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

Abstract-We consider a decentralized optimal control problem for a linear plant controlled by two controllers, a local controller and a remote controller. The local controller directly observes the state of the plant and can inform the remote controller of the plant state through a packet-drop channel. We assume that the remote controller is able to send acknowledgments to the local controller to signal the successful receipt of transmitted packets. The objective of the two controllers is to cooperatively minimize a quadratic performance cost. We provide a dynamic program for this decentralized control problem using the common information approach. Although our problem is not a partially nested LQG problem, we obtain explicit optimal strategies for the two controllers. In the optimal strategies, both controllers compute a common estimate of the plant state based on the common information. The remote controller's action is linear in the common estimated state, and the local controller's action is linear in both the actual state and the common estimated state.

show abstract

Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

Cited by 240 publications

References 39 publications

Game Theory and Control

Game Theory and Control

Multi-objective decision-theoretic planning

Optimal local and remote controllers with unreliable communication

Contact Info

Product

Resources

About