A Survey of Approximate Dynamic Programming

Wang, Lin; Peng, Hui; Zhu, Huayong

doi:10.1109/ihmsc.2009.222

Cited by 9 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accounting for the future in this way can involve a considerable computing cost (Marescot et al, 2013 ). However, advances in finding efficient solution approaches to dynamic decision problems (Wang et al, 2009 ) have extended the range of applicability in ecology and greatly increased our ability to identify effective strategies for long‐term conservation.…”

Section: Dynamic Approach To the Four Conservation Challengesmentioning

confidence: 99%

Four conservation challenges and a synthesis

Williams

Brown

2023

Ecology and Evolution

View full text Add to dashboard Cite

Conservation and management of biological systems involves decision‐making over time, with a generic goal of sustaining systems and their capacity to function in the future. We address four persistent and difficult conservation challenges: (1) prediction of future consequences of management, (2) uncertainty about the system's structure, (3) inability to observe ecological systems fully, and (4) nonstationary system dynamics. We describe these challenges in terms of dynamic systems subject to different sources of uncertainty, and we present a basic Markovian framework that can encompass approaches to all four challenges. Finding optimal conservation strategies for each challenge requires issue‐specific structural features, including adaptations of state transition models, uncertainty metrics, valuation of accumulated returns, and solution methods. Strategy valuation exhibits not only some remarkable similarities among approaches but also some important operational differences. Technical linkages among the models highlight synergies in solution approaches, as well as possibilities for combining them in particular conservation problems. As methodology and computing software advance, such an integrated conservation framework offers the potential to improve conservation outcomes with strategies to allocate management resources efficiently and avoid negative consequences.

show abstract

Section: Dynamic Approach To the Four Conservation Challengesmentioning

confidence: 99%

Four conservation challenges and a synthesis

Williams

Brown

2023

Ecology and Evolution

View full text Add to dashboard Cite

show abstract

“…It is well known that MDP's have efficient solutions using dynamic programming. 10 On the other hand, computing optimal solutions to POMDP's can be undecidable in some situations, meaning there can be no algorithms for computing optimal solutions in finite, or even exponential, time. 11 Our goal is to develop metrics that measure the extent to which Red's decisions become more complex due to:…”

Section: Introductionmentioning

confidence: 99%

Quantifying decision complexity in IADS operations

Sheldon,

Hou,

Bouillet

et al. 2024

Disruptive Technologies in Information Sciences VIII

View full text Add to dashboard Cite

Decision Advantage is a goal in current and future military operations. Achieving such an advantage can be done by degrading adversaries' decision making ability through imposition of complexity into the decision problems they have to make. This paper describes mathematical techniques for quantifying decision complexity in Integrated Air Defense Systems (IADS). The methods are based on graph properties derived from the defender's IADS' System of Systems description and the attacker's Course of Action (COA) plans. Multiple plans can be compared quantitatively with respect to the decision complexity they impose on the defender. using metrics that are semantically meaningful to planners. The metrics developed are able to expose subtle ways that COAs impose complexity on an adversary, that may not be obvious to an operational planner at first glance. Keywords:"The quality of decision is like the well-timed swoop of a falcon which enables it to strike and destroy its victim." Sun Tzu

show abstract

“…the fuzzy dynamic programming proposed in Abo-Sinna (2004). To break the 'curse-of-dimensionality' associated with the traditional dynamic programming approach, approximate dynamic programming is developed by combining simulation and function approximation (Wang et al, 2009), where heuristic dynamic programming (HDP) (Werbos, 1977), action-dependent HDP (Werbos, 1989), neuro-dynamic programming (Bertsekas & Tsitsiklis, 1995) and learning-based algorithms (Lewis & Vrabie, 2009;Tsitsiklis & Roy, 1997) can be used.…”

Section: Introductionmentioning

confidence: 99%

High-order fully actuated system approaches: Part VIII. Optimal control with application in spacecraft attitude stabilisation

Duan

2021

International Journal of Systems Science

103

View full text Add to dashboard Cite

In this paper, the optimal control problem for dynamical systems represented by general highorder fully actuated (HOFA) models is formulated. The problem aims to minimise an objective in the quadratic form of the states and their derivatives of certain orders. The designed controller is a combination of the linearising nonlinear controller and an optimal quadratic controller for a converted linear system. In the infinite-time output regulation case, the solution is in essence a nonlinear state feedback dependent on a well-known Riccati algebraic equation. In the sub-fully actuated system case, the feasibility of the controller is investigated and guaranteed by properly characterising a ball restriction area of the system initial values. Application of the optimal control technique for sub-fully actuated systems to a spacecraft attitude control provides very smooth and steady responses and well demonstrates the effect and simplicity of the proposed approach.

show abstract

A Survey of Approximate Dynamic Programming

Cited by 9 publications

References 29 publications

Four conservation challenges and a synthesis

Four conservation challenges and a synthesis

Quantifying decision complexity in IADS operations

High-order fully actuated system approaches: Part VIII. Optimal control with application in spacecraft attitude stabilisation

Contact Info

Product

Resources

About