Evolutionary value function approximation

Davarynejad, Mohsen; Ast, Jelmer van; Vrancken, Jos; Berg, Jan van den

doi:10.1109/adprl.2011.5967349

Cited by 3 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Global parametric function approximation can be realized with ANNs [5] or Evolutionary Value Function Approximations [15], combining Reinforcement Learning with Genetic Algorithms. In contrast, local nonparametric function approximation is represented by instance-based algorithms in terms of classical value tables with k-NN prediction [13,16] and kernel regression [5].…”

Section: Related Workmentioning

confidence: 99%

Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulations

Senn

Link

Pollak

et al. 2014

Journal of Process Control

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulations

Senn

Link

Pollak

et al. 2014

Journal of Process Control

View full text Add to dashboard Cite

“…To our best knowledge, there have been no reports in the literature on the use of symbolic regression for constructing value functions. The closest related research is the use of genetic programming for fitting already available V-functions [17], [18], which, however, is completely different from our approach.…”

Section: Introductionmentioning

confidence: 99%

Symbolic Regression Methods for Reinforcement Learning

Kubalı́k

Derner²,

Žegklitz

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Reinforcement learning algorithms can be used to optimally solve dynamic decision-making and control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as neural networks or basis function expansions, have two main drawbacks: they are black-box models offering no insight in the mappings learned, and they require significant trial and error tuning of their meta-parameters. In this paper, we propose a new approach to constructing smooth value functions in the form of analytic expressions by means of symbolic regression. We introduce three off-line methods for finding value functions based on a state transition model: symbolic value iteration, symbolic policy iteration, and a direct solution of the Bellman equation. The methods are illustrated on four nonlinear control problems: velocity control under friction, one-link and two-link pendulum swing-up, and magnetic manipulation. The results show that the value functions not only yield well-performing policies, but also are compact, mathematically tractable and easy to plug into other algorithms. This makes them potentially suitable for further analysis of the closed-loop system. A comparison with an alternative approach using neural networks shows that our method outperforms the neural network-based one.

show abstract

Symbolic method for deriving policy in reinforcement learning

Alibekov

Kubalı́k

Babuška

2016

2016 IEEE 55th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Abstract-This paper addresses the problem of deriving a policy from the value function in the context of reinforcement learning in continuous state and input spaces. We propose a novel method based on genetic programming to construct a symbolic function, which serves as a proxy to the value function and from which a continuous policy is derived. The symbolic proxy function is constructed such that it maximizes the number of correct choices of the control input for a set of selected states. Maximization methods can then be used to derive a control policy that performs better than the policy derived from the original approximate value function. The method was experimentally evaluated on two control problems with continuous spaces, pendulum swing-up and magnetic manipulation, and compared to a standard policy derivation method using the value function approximation. The results show that the proposed method and its variants outperform the standard method.

show abstract

Evolutionary value function approximation

Cited by 3 publications

References 14 publications

Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulations

Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulations

Symbolic Regression Methods for Reinforcement Learning

Symbolic method for deriving policy in reinforcement learning

Contact Info

Product

Resources

About