Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC*

Bertsekas, Dimitri P.

doi:10.3166/ejc.11.310-334

Cited by 338 publications

(219 citation statements)

References 35 publications

Supporting

Mentioning

217

Contrasting

Unclassified

Order By: Relevance

“…To overcome the burden of dimensionality, in this paper, we use MPC [see Bertsekas, 2005;Scattolini, 2009, and references therein], a real-time control approach based on the sequential resolution of multiple open-loop control problems defined over a finite, receding time horizon [Mayne et al, 2000]. At each time t, a forecast of the external drivers (e.g., the inflow), called nominal value, is provided over the finite horizon t; t þ h ½ by a predictor that uses all the information available at time t (e.g., precipitation and inflow at previous time).…”

Section: Fully Cooperative and Informative Scenariomentioning

confidence: 99%

“…[10] The optimization of the agents' decisions is done according to a model predictive control (MPC) scheme [see Bertsekas, 2005], which is particularly suitable for largescale systems as well as for decentralized control strategies, overcoming the curse of dimensionality that limits the applicability of classical control techniques such as stochastic dynamic programming and approximated approaches [e.g., Castelletti et al, 2010]. Though being largely adopted in process-engineering problems [Scattolini, 2009], MPC is nearly unexplored in the water resources management [Niewiadomska-Szynkiewicz et al, 1996;Negenborn et al, 2009;Anand et al, 2011] and especially in noncooperative settings.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Assessing the value of cooperation and information exchange in large water resources systems by agent‐based optimization

Giuliani

Castelletti

2013

Water Resources Research

View full text Add to dashboard Cite

[1] Many large-scale water resources systems, especially in transboundary contexts, are characterized by the presence of several and conflicting interests and managed by multiple, institutionally independent decision makers. These systems are often studied adopting a centralized approach based on the assumption of full cooperation and information exchange among the involved parties. Such a perspective is conceptually interesting to quantify the best achievable performance but might have little practical impact given the real political and institutional setting. In this work, we propose a novel decision-analytic framework based on multiagent systems to model and analyze different levels of cooperation and information exchange among multiple decision makers. The Zambezi River basin is used as a case study. According to the proposed agent-based optimization approach, each agent represents a decision maker, whose decisions are defined by an explicit optimization problem considering only the agent's local interests. The economic value of information exchange is estimated comparing a noncooperative setting, where agents act independently, with the first basic level of cooperation, i.e., coordination, characterized by full information exchange. The economic value of cooperation is also estimated by comparison with the ideal, fully cooperative management of the system. Results show that coordination, obtained with complete information exchange, allows the downstream agents to better adapt to the upstream behaviors. The impact of information exchange depends on the objective considered, and we show coordination to be particularly beneficial to environmental interests.Citation: Giuliani, M., and A. Castelletti (2013), Assessing the value of cooperation and information exchange in large water resources systems by agent-based optimization, Water Resour. Res., 49,[3912][3913][3914][3915][3916][3917][3918][3919][3920][3921][3922][3923][3924][3925][3926]

show abstract

Section: Fully Cooperative and Informative Scenariomentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Assessing the value of cooperation and information exchange in large water resources systems by agent‐based optimization

Giuliani

Castelletti

2013

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…In constrained continuous dynamic simulation, two basic methodologies for solving a finite horizon NLP problem exist: sequential methods and simultaneous methods [25], although other methods, including hybrids of the two (i.e., multiple shooting methods) may also be used [26,27]. Sequential and simultaneous methods are briefly introduced in Sections 2.2 and 2.3.…”

Section: Logical Disjunctions In Optimizationmentioning

confidence: 99%

A Continuous Formulation for Logical Decisions in Differential Algebraic Systems using Mathematical Programs with Complementarity Constraints

et al. 2016

View full text Add to dashboard Cite

This work presents a methodology to represent logical decisions in differential algebraic equation simulation and constrained optimization problems using a set of continuous algebraic equations. The formulations may be used when state variables trigger a change in process dynamics, and introduces a pseudo-binary decision variable, which is continuous, but should only have valid solutions at values of either zero or one within a finite time horizon. This formulation enables dynamic optimization problems with logical disjunctions to be solved by simultaneous solution methods without using methods such as mixed integer programming. Several case studies are given to illustrate the value of this methodology including nonlinear model predictive control of a chemical reactor using a surge tank with overflow to buffer disturbances in feed flow rate. Although this work contains novel methodologies for solving dynamic algebraic equation (DAE) constrained problems where the system may experience an abrupt change in dynamics that may otherwise require a conditional statement, there remain substantial limitations to this methodology, including a limited domain where problems may converge and the possibility for ill-conditioning. Although the problems presented use only continuous algebraic equations, the formulation has inherent non-smoothness. Hence, these problems must be solved with care and only in select circumstances, such as in simulation or situations when the solution is expected to be near the solver's initial point.

show abstract

“…MPC has been studied extensively (see reviews by Morari and Lee [16], Bertsekas [4], Mayne et al [14], Diehl et al [8], and references therein), but its application to robotic domains is only starting to gain popularity [1]. Chen and Allgöwer [7] seem to be the first to suggest the combination of MPC with an infinite-horizon optimization problem, and this approach has been studied extensively in the past decade [18,2,11].…”

Section: Related Workmentioning

confidence: 99%

Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts

Erez

Tassa

Todorov

2011

Robotics: Science and Systems VII

View full text Add to dashboard Cite

Abstract-We present a method that combines offline trajectory optimization and online Model Predictive Control (MPC), generating robust controllers for complex periodic behavior in domains with unilateral constraints (e.g., contact with the environment). MPC offers robust and adaptive control even in high-dimensional domains; however, the online optimization gets stuck in local minima when the domains has discontinuous dynamics. Some methods of trajectory optimization that are immune to such problems, but these are often too slow to be applied online.In this paper, we use offline optimization to find the limit-cycle solution of an infinite-horizon average-cost optimal-control task. We then compute a local quadratic approximation of the Value function around this limit cycle. Finally, we use this quadratic approximation as the terminal cost of an online MPC.This combination of an offline solution of the infinite-horizon problem with an online MPC controller is known as Infinite Horizon Model Predictive Control (IHMPC), and has previously been applied only to simple stabilization objectives. Here we extend IHMPC to tackle periodic tasks, and demonstrate the power of our approach by synthesizing hopping behavior in a simulated robot. IHMPC involves a limited computational load, and can be executed online on a standard laptop computer. The resulting behavior is extremely robust, allowing the hopper to recover from virtually any perturbation.In real robotic domains, modeling errors are inevitable. We show how IHMPC is robust to modeling errors by altering the morphology of the robot; the same controller remains effective, even when the underlying infinite-horizon solution is no longer accurate.

show abstract

Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC*

Cited by 338 publications

References 35 publications

Assessing the value of cooperation and information exchange in large water resources systems by agent‐based optimization

Assessing the value of cooperation and information exchange in large water resources systems by agent‐based optimization

A Continuous Formulation for Logical Decisions in Differential Algebraic Systems using Mathematical Programs with Complementarity Constraints

Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts

Contact Info

Product

Resources

About