Linearly Solvable Optimal Control

Dvijotham, Krishnamurthy; Todorov, Emanuel

doi:10.1002/9781118453988.ch6

Cited by 42 publications

(41 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a surprisingly high bar. For example, while many Bayesian or optimal control algorithms are used to control robots [21], we find few neuromorphic implementations of such algorithms. Lastly, building Bayesian models of information integration leads to an understanding of the relevant variables and their interplay.…”

Section: Assuming That the Brain Solves Problems Close To The Bayesiamentioning

confidence: 99%

Bayesian statistics: relevant for the brain?

Kording¹

2014

Current Opinion in Neurobiology

View full text Add to dashboard Cite

Analyzing data from experiments involves variables that we neuroscientists are uncertain about. Efficiently calculating with such variables usually requires Bayesian statistics. As it is crucial when analyzing complex data, it seems natural that the brain would “use” such statistics to analyze data from the world. And indeed, recent studies in the areas of perception, action, and cognition suggest that Bayesian behavior is widespread, in many modalities and species. Consequently, many models have suggested that the brain is built on simple Bayesian principles. While the brain’s code is probably not actually simple, I believe that Bayesian principles will facilitate the construction of faithful models of the brain.

show abstract

Section: Assuming That the Brain Solves Problems Close To The Bayesiamentioning

confidence: 99%

Bayesian statistics: relevant for the brain?

Kording¹

2014

Current Opinion in Neurobiology

View full text Add to dashboard Cite

show abstract

“…Eq. (1.4) belongs to the family of the so-called LS-MDPs introduced in [39], discussed in [19,20,32], and briefly described as a special case in Appendix 1.9. Solution of Eqs.…”

Section: Cost-vs-welfare Optimalmentioning

confidence: 99%

Ensemble Control of Cycling Energy Loads: Markov Decision Approach

Chertkov

Chernyak

Deka

2018

Energy Markets and Responsive Grids

View full text Add to dashboard Cite

A Markov decision process (MDP) framework is adopted to represent ensemble control of devices with cyclic energy consumption patterns, e.g., thermostatically controlled loads. Specifically we utilize and develop the class of MDP models previously coined linearly solvable MDPs, that describe optimal dynamics of the probability distribution of an ensemble of many cycling devices. Two principally different settings are discussed. First, we consider optimal strategy of the ensemble aggregator balancing between minimization of the cost of operations and minimization of the ensemble welfare penalty, where the latter is represented as a KL-divergence between actual and normal probability distributions of the ensemble. Then, second, we shift to the demand response setting modeling the aggregator's task to minimize the welfare penalty under the condition that the aggregated consumption matches the targeted time-varying consumption requested by the system operator. We discuss a modification of both settings aimed at encouraging or constraining the transitions between different states. The dynamic programming feature of the resulting modified MDPs is always preserved; however, 'linear solvability' is lost fully or partially, depending on the type of modification. We also conducted some (limited in scope) numerical experimentation using the formulations of the first setting. We conclude by discussing future generalizations and applications.

show abstract

“…The result in this section will serve as a tool to find an MFE in the road traffic game described above. The main emphasis in this section is that the introduced auxiliary optimal control problem belongs to the class of linearly-solvable MDPs [9], [25]. This fact provides a tremendous advantage in the computation of mean-field equilibria in the road traffic game.…”

Section: Linearly Solvable Mdpsmentioning

confidence: 99%

Linearly Solvable Mean-Field Road Traffic Games

Tanaka

Nekouei

Johansson

2018

2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

We analyze the behavior of a large number of strategic drivers traveling over an urban traffic network using the mean-field game framework. We assume an incentive mechanism for congestion mitigation under which each driver selecting a particular route is charged a tax penalty that is affine in the logarithm of the number of agents selecting the same route. We show that the mean-field approximation of such a large-population dynamic game leads to the so-called linearly solvable Markov decision process, implying that an open-loop -Nash equilibrium of the original game can be found simply by solving a finite-dimensional linear system.

show abstract

Linearly Solvable Optimal Control

Cited by 42 publications

References 25 publications

Bayesian statistics: relevant for the brain?

Bayesian statistics: relevant for the brain?

Ensemble Control of Cycling Energy Loads: Markov Decision Approach

Linearly Solvable Mean-Field Road Traffic Games

Contact Info

Product

Resources

About