Adaptive Dynamic Programming: An Introduction

Wang, Fei‐Yue; Zhang, Huaguang; Liu, Derong

doi:10.1109/mci.2009.932261

Cited by 743 publications

(264 citation statements)

References 89 publications

Supporting

Mentioning

262

Contrasting

Unclassified

Order By: Relevance

“…These neural-network approaches have mainly been based on Adaptive Critic Designs (ACDs) (Wang et al, 2009;Prokhorov and Wunsch, 1997;Werbos, 1992). ACDs use two neural networks: an action network and a critic network.…”

Section: Introductionmentioning

confidence: 99%

An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances

Fairbank¹,

Li²,

Fu³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link AbstractWe present a recurrent neural-network (RNN) controller designed to solve the tracking problem for control systems. We demonstrate that a major difficulty in training any RNN is the problem of exploding gradients, and we propose a solution to this in the case of tracking problems, by introducing a stabilization matrix and by using carefully constrained context units. This solution allows us to achieve consistently lower training errors, and hence allows us to more easily introduce adaptive capabilities. The resulting RNN is one that has been trained off-line to be rapidly adaptive to changing plant conditions and changing tracking targets.The case study we use is a renewable-energy generator application; that of producing an efficient controller for a three-phase grid-connected converter. The controller we produce can cope with random variation of system parameters and fluctuating grid voltages. It produces tracking control with almost instantaneous response to changing reference states, and virtually zero oscil-$ This work was supported in part by the U.S. National Science Foundation under Grant EECS 1059265/1102159, the Mary K. Finley Missouri Endowment, and the Missouri S&T Intelligent Systems Center.Email addresses: michael.fairbank@virgin.net (Michael Fairbank), sli@eng.ua.edu (Shuhui Li), xfu@crimson.ua.edu (Xingang Fu), E.Alonso@city.ac.uk (Eduardo Alonso), dwunsch@mst.edu (Donald Wunsch) Preprint submitted to Neural Networks September 23, 2013 lation. This compares very favorably to the classical proportional integrator (PI) controllers, which we show produce a much slower response and settling time. In addition, the RNN we propose exhibits better learning stability and convergence properties, and can exhibit faster adaptation, than is achievable with adaptive critic designs.

show abstract

Section: Introductionmentioning

confidence: 99%

An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances

Fairbank¹,

Li²,

Fu³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

show abstract

“…real-time) for adapting the management of dynamic systems is the essence of many dynamic/adaptive control strategies [55,56]. Some strategies allow varying uncertainties -even related to datadriven information -to be explicitely modelled.…”

Section: Uncertainty and Complexity: Semantic Array Programmingmentioning

confidence: 99%

“…Accurate descriptions of complex systems often imply wider dimensionality of the state vector x. Several strategies [56,61] support natural resources applications in mitigating the intractability of complex problems, by e.g. approximating the SDP with modified algorithms [24,62,63].…”

Section: Uncertainty and Complexity: Semantic Array Programmingmentioning

confidence: 99%

An Architecture for Adaptive Robust Modelling of Wildfire Behaviour under Deep Uncertainty

Rigo

Rodriguez-Aseretto

Bosco

et al. 2013

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. Wildfires in Europe -especially in the Mediterranean region -are one of the major treats at landscape scale. While their immediate impact ranges from endangering human life to the destruction of economic assets, other damages exceed the spatio-temporal scale of a fire event. Wildfires involving forest resources are associated with intense carbon emissions and alteration of surrounding ecosystems. The induced land cover degradation has also a potential role in exacerbating soil erosion and shallow landslides. A component of the complexity in assessing fire impacts resides in the difference between uncontrolled wildfires and those for which a control strategy is applied. Robust modelling of wildfire behaviour requires dynamic simulations under an array of multiple fuel models, meteorological disturbances and control strategies for mitigating fire damages. Uncertainty is associated to meteorological forecast and fuel model estimation. Software uncertainty also derives from the datatransformation models needed for predicting the wildfire behaviour and its consequences. The complex and dynamic interactions of these factors define a context of deep uncertainty. Here an architecture for adaptive and robust modelling of wildfire behaviour is proposed, following the semantic array programming paradigm. The mathematical conceptualisation focuses on the dynamic exploitation of updated meteorological information and the design flexibility in adapting to the heterogeneous European conditions. Also, the modelling architecture proposes a multicriteria approach for assessing the potential impact with qualitative rapid assessment methods and more accurate a-posteriori assessment.

show abstract

“…VGL is an extension of well-known methods in adaptive dynamic programming, Dual Heuristic Programming and Generalized Dual Heuristic Programming in particular, that have been proved to be successful in solving complex tasks such as autopilot landing, power system control, simple control benchmark problems such as "pole balancing", and many others (Wang, Zhang & Liu 2009 …”

Section: Value-gradient Learning and Temporal Differencementioning

confidence: 99%